2024-03-23T23:19:02.778 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T23:19:02.778 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393 --owner mchangir --verbose --name mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/orig.config.yaml 2024-03-23T23:19:02.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T23:19:02.982 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} email: mchangir@redhat.com first_in_suite: false job_id: '7616393' kernel: client: branch: testing flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3550 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 613/1024 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4UZn3fzouSCKfvtBoeO0x+s9TZX/BRQVoHoDAsAWjVY37BVvbBlS2hn2A0XKBXK09iBkGlHyCD8k0Q0WOGyQc= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2OXbuRcMHP6jEp4VCLlldmKzvVZHylexlT3WdnUEYRmPOnDPk7g3E2ykjebN6SHp8IKLkM9wMhWfbUfmhLemA= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUAe1z2NFyLdfNY3eaJkHruYWyySMxXI+yKD+SaVX2VXOa/5AMqzJVUtF3d2ZWsLcQZDqZgQfb2IqL7gkNTY4Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_09:49:57 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T23:19:02.982 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-23T23:19:02.983 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-23T23:19:02.983 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T23:19:03.086 INFO:teuthology.task.internal:Checking packages... 2024-03-23T23:19:03.112 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-23T23:19:03.113 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T23:19:03.113 INFO:teuthology.packaging:ref: None 2024-03-23T23:19:03.113 INFO:teuthology.packaging:tag: None 2024-03-23T23:19:03.113 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-23T23:19:03.113 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:19:03.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-23T23:19:03.344 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-23T23:19:03.345 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T23:19:03.352 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T23:19:03.352 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T23:19:03.360 INFO:teuthology.task.internal:Saving configuration 2024-03-23T23:19:03.379 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T23:19:03.385 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T23:19:03.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.272089', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4UZn3fzouSCKfvtBoeO0x+s9TZX/BRQVoHoDAsAWjVY37BVvbBlS2hn2A0XKBXK09iBkGlHyCD8k0Q0WOGyQc='} 2024-03-23T23:19:03.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.269533', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2OXbuRcMHP6jEp4VCLlldmKzvVZHylexlT3WdnUEYRmPOnDPk7g3E2ykjebN6SHp8IKLkM9wMhWfbUfmhLemA='} 2024-03-23T23:19:03.452 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.270962', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUAe1z2NFyLdfNY3eaJkHruYWyySMxXI+yKD+SaVX2VXOa/5AMqzJVUtF3d2ZWsLcQZDqZgQfb2IqL7gkNTY4Y='} 2024-03-23T23:19:03.453 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T23:19:03.459 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-23T23:19:03.459 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-23T23:19:03.459 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-23T23:19:03.459 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T23:19:03.561 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23f4701f70>, signals=[15]) 2024-03-23T23:19:03.562 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T23:19:03.568 INFO:teuthology.task.internal:Opening connections... 2024-03-23T23:19:03.568 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:19:03.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:19:03.665 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-03-23T23:19:03.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:19:03.755 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-03-23T23:19:03.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:19:03.844 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T23:19:03.851 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-03-23T23:19:03.869 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-03-23T23:19:03.869 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-03-23T23:19:03.926 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-03-23T23:19:03.926 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-03-23T23:19:03.926 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T23:19:03.927 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T23:19:03.928 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-03-23T23:19:03.956 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-03-23T23:19:03.974 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-03-23T23:19:03.974 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-03-23T23:19:04.030 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T23:19:04.031 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T23:19:04.032 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T23:19:04.032 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-03-23T23:19:04.055 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-03-23T23:19:04.073 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-03-23T23:19:04.074 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-03-23T23:19:04.130 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T23:19:04.131 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T23:19:04.131 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-03-23T23:19:04.156 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T23:19:04.167 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T23:19:04.172 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T23:19:04.173 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-03-23T23:19:04.175 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-03-23T23:19:04.178 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-03-23T23:19:04.194 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T23:19:04.200 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T23:19:04.200 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-03-23T23:19:04.232 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-03-23T23:19:04.235 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-03-23T23:19:04.326 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T23:19:04.407 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T23:19:04.423 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-23T23:19:04.423 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-23T23:19:04.423 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-23T23:19:04.423 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T23:19:04.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:19:04.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-23T23:19:04.764 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T23:19:04.764 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-23T23:19:04.765 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-23T23:19:04.782 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-361.el9.x86_64 2024-03-23T23:19:04.782 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T23:19:04.782 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-23T23:19:04.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2024-03-23T23:19:08.225 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - AppStream 3.8 MB/s | 6.7 MB 00:01 2024-03-23T23:19:09.817 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-23T23:19:10.991 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-03-23T23:19:12.659 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-03-23T23:19:20.358 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.1 MB 00:04 2024-03-23T23:19:21.925 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-23T23:19:23.118 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-23T23:19:28.160 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-23T23:19:30.292 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-23T23:19:30.302 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-23T23:19:30.302 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 1 Package 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 375 M 2024-03-23T23:19:30.303 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-03-23T23:19:38.202 INFO:teuthology.orchestra.run.smithi099.stdout:linux-firmware-20240219-143.el9.noarch.rpm 47 MB/s | 375 MB 00:07 2024-03-23T23:19:38.202 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-23T23:19:38.203 INFO:teuthology.orchestra.run.smithi099.stdout:Total 47 MB/s | 375 MB 00:07 2024-03-23T23:19:38.203 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-03-23T23:19:38.294 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-03-23T23:19:38.294 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-03-23T23:19:40.654 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-03-23T23:19:40.655 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-03-23T23:19:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-03-23T23:19:47.303 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T23:19:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T23:19:48.664 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T23:19:48.664 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-23T23:19:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-23T23:19:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:19:49.036 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-03-23T23:19:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-23T23:19:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:19:49.037 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-23T23:19:49.313 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T23:19:49.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-23T23:19:49.460 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-23T23:19:49.461 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-23T23:19:49.463 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-03-23T23:20:00.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23 23:20:00 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-23T23:20:00.860 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2024-03-23 23:20:00-- 2024-03-23T23:20:00.860 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 11s 2024-03-23T23:20:00.860 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-23T23:20:00.863 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on client.0... 2024-03-23T23:20:00.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-23T23:20:02.795 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2024-03-23T23:20:02.868 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2024-03-23T23:20:02.947 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2024-03-23T23:20:10.516 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-23T23:20:27.205 INFO:teuthology.orchestra.run.smithi099.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T23:20:49.604 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-23T23:20:49.605 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-23T23:20:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-23T23:20:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-23T23:20:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-23T23:20:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-23T23:20:49.695 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-23T23:20:49.696 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-23T23:20:49.697 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-23T23:20:49.698 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-23T23:20:49.699 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-23T23:20:49.700 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-23T23:20:49.701 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-23T23:20:49.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-23T23:20:49.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-23T23:20:49.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-23T23:20:49.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-23T23:20:49.702 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-23T23:20:49.703 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-23T23:20:49.704 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-23T23:20:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-23T23:20:49.706 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-23T23:20:49.707 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-23T23:20:49.708 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-23T23:20:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-23T23:20:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-23T23:20:49.711 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-23T23:20:49.712 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-23T23:20:49.713 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-23T23:20:49.714 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-23T23:20:49.715 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-23T23:20:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-23T23:20:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-23T23:20:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-23T23:20:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-23T23:20:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-23T23:20:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-23T23:20:49.722 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-23T23:20:49.723 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-23T23:20:49.724 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-23T23:20:49.725 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-23T23:20:49.726 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-23T23:20:49.727 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-23T23:20:49.728 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-23T23:20:49.729 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-23T23:20:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-23T23:20:49.731 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-23T23:20:49.732 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-23T23:20:49.733 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-23T23:20:49.734 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-23T23:20:49.735 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-23T23:20:49.738 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-23T23:20:49.739 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-23T23:20:49.740 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-23T23:20:49.741 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-23T23:20:49.742 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-23T23:20:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-23T23:20:49.744 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-23T23:20:49.745 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-23T23:20:49.746 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-23T23:20:49.747 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-23T23:20:49.748 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-23T23:20:49.749 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-23T23:20:49.750 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-23T23:20:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-23T23:20:49.752 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-23T23:20:49.753 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-23T23:20:49.754 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-23T23:20:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-23T23:20:49.756 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-23T23:20:49.757 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-23T23:20:49.758 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-23T23:20:49.759 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-23T23:20:49.760 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-23T23:20:49.761 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-23T23:20:49.762 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-23T23:20:49.763 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-23T23:20:49.764 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-23T23:20:49.765 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-23T23:20:49.766 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-23T23:20:49.767 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-23T23:20:49.768 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-23T23:20:49.769 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-23T23:20:49.770 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-23T23:20:49.771 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-23T23:20:49.772 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-23T23:20:49.773 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-23T23:20:49.774 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-23T23:20:49.775 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-23T23:20:49.776 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-23T23:20:49.777 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-23T23:20:49.778 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-23T23:20:49.779 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-23T23:20:49.780 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-23T23:20:49.781 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-23T23:20:49.782 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-23T23:20:49.783 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-23T23:20:49.784 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-23T23:20:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-23T23:20:49.786 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-23T23:20:49.787 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-23T23:20:49.788 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-23T23:20:49.789 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-23T23:20:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-23T23:20:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-23T23:20:49.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-23T23:20:49.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-23T23:20:49.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-23T23:20:49.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-23T23:20:49.792 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-23T23:20:49.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-23T23:20:49.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-23T23:20:49.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-23T23:20:49.793 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-23T23:20:49.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-23T23:20:49.794 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-23T23:20:49.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-23T23:20:49.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-23T23:20:49.795 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-23T23:20:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-23T23:20:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-23T23:20:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-23T23:20:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-23T23:20:49.796 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-23T23:20:49.797 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-23T23:20:49.798 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-23T23:20:49.799 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-23T23:20:49.800 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-23T23:20:49.801 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-23T23:20:49.802 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-23T23:20:49.803 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-23T23:20:49.804 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-23T23:20:49.805 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-23T23:20:49.806 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-23T23:20:49.807 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-23T23:20:49.808 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-23T23:20:49.809 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-23T23:20:49.810 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-23T23:20:49.811 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-23T23:20:49.812 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-23T23:20:49.813 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-23T23:20:49.814 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-23T23:20:49.815 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-23T23:20:49.816 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-23T23:20:49.817 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-23T23:20:49.818 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-23T23:20:49.819 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-23T23:20:49.820 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-23T23:20:49.821 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-23T23:20:49.822 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-23T23:20:49.823 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-23T23:20:49.824 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-23T23:20:49.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-23T23:20:49.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-23T23:20:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-23T23:20:49.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-23T23:20:49.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-23T23:20:49.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-23T23:20:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-23T23:20:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-23T23:20:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-23T23:20:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-23T23:20:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-23T23:20:49.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-23T23:20:49.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-23T23:20:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-23T23:20:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-23T23:20:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-23T23:20:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-23T23:20:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-23T23:20:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-23T23:20:49.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-23T23:20:49.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-23T23:20:49.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-23T23:20:49.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-23T23:20:49.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-23T23:20:49.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-23T23:20:49.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-23T23:20:49.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-23T23:20:49.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-23T23:20:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-23T23:20:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-23T23:20:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-23T23:20:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-23T23:20:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-23T23:20:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-23T23:20:49.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-23T23:20:49.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-23T23:20:49.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-23T23:20:49.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-23T23:20:49.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-23T23:20:49.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-23T23:20:49.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-23T23:20:49.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-23T23:20:49.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-23T23:20:49.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-23T23:20:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-23T23:20:49.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-23T23:20:49.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-23T23:20:49.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-23T23:20:49.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-23T23:20:49.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-23T23:20:49.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-23T23:20:49.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-23T23:20:49.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-23T23:20:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-23T23:20:49.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-23T23:20:49.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-23T23:20:49.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-23T23:20:49.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-23T23:20:49.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-23T23:20:49.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-23T23:20:49.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-23T23:20:49.866 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-23T23:20:49.867 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-23T23:20:49.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-23T23:20:49.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-23T23:20:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-23T23:20:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-23T23:20:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-23T23:20:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-23T23:20:49.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-23T23:20:49.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-23T23:20:49.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-23T23:20:49.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-23T23:20:49.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-23T23:20:49.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-23T23:20:49.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-23T23:20:49.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-23T23:20:49.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-23T23:20:49.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-23T23:20:49.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-23T23:20:49.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-23T23:20:49.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-23T23:20:49.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-23T23:20:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T23:20:49.893 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:49.893 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-23T23:20:49.916 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2024-03-23T23:20:49.916 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-23T23:20:49.916 INFO:teuthology.orchestra.run.smithi099.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:for file in vmlinuz System.map config; do 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout: fi 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-23T23:20:49.917 INFO:teuthology.orchestra.run.smithi099.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-23T23:20:49.918 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-23T23:20:49.918 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:49.918 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-23T23:20:49.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-03-23T23:20:49.996 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 68.el9 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8264254 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T23:20:49.997 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-03-23T23:20:49.998 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:20:49.999 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T23:20:49.999 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T23:20:50.000 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:50.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T23:20:50.627 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-03-23T23:20:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T23:20:52.115 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-03-23T23:20:52.118 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-03-23T23:20:52.133 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.dkPW6yU0hj 2024-03-23T23:20:52.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dkPW6yU0hj 2024-03-23T23:20:52.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.dkPW6yU0hj 2024-03-23T23:20:52.356 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.dkPW6yU0hj is 6KB 2024-03-23T23:20:52.407 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.dkPW6yU0hj 2024-03-23T23:20:52.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T23:20:52.485 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T23:20:52.486 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T23:20:52.486 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T23:20:52.486 INFO:teuthology.orchestra.run.smithi099.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-23T23:20:52.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-23T23:20:52.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-03-23T23:21:22.629 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-23T23:21:22.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:21:22.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:21:41.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-23T23:21:50.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:21:50.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:21:53.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-23T23:22:05.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:22:05.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:22:23.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-23T23:22:38.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:22:38.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:22:40.660 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-23T23:22:41.340 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:22:41.340 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g5ec3e6e88b2e"... 2024-03-23T23:22:41.341 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-23T23:22:41.378 INFO:teuthology.orchestra.run.smithi099.stdout:6.8.0-g5ec3e6e88b2e 2024-03-23T23:22:41.379 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-23T23:22:41.379 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T23:22:41.379 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T23:22:42.380 DEBUG:teuthology.parallel:result is None 2024-03-23T23:22:42.380 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T23:22:42.388 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T23:22:42.389 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T23:22:42.394 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T23:22:42.397 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T23:22:42.433 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T23:22:42.439 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T23:22:42.448 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T23:22:42.448 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T23:22:42.477 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T23:22:42.480 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T23:22:42.633 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T23:22:42.640 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T23:22:42.640 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T23:22:42.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:22:42.657 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T23:22:42.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:22:42.674 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T23:22:42.679 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T23:22:42.700 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T23:22:42.735 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.759 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.764 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.783 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.808 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.903 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T23:22:42.909 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T23:22:42.916 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T23:22:42.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T23:22:42.921 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T23:22:42.923 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T23:22:43.001 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T23:22:43.010 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T23:22:43.010 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T23:22:43.014 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T23:22:43.017 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T23:22:43.051 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:43.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:43.251 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:43.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:43.393 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:22:43.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T23:22:43.486 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:43.531 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:43.587 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:43.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:43.714 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:22:43.714 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T23:22:43.783 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:43.828 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T23:22:44.119 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:44.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T23:22:44.209 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:22:44.209 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T23:22:44.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-03-23T23:22:44.288 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-03-23T23:22:44.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-03-23T23:22:44.334 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:22:44.373 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:22:44.405 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:22:44.742 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T23:22:44.750 INFO:teuthology.task.internal:Starting timer... 2024-03-23T23:22:44.750 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T23:22:44.790 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T23:22:44.799 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-23T23:22:44.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-03-23T23:22:44.907 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-03-23T23:22:44.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-03-23T23:22:44.971 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-03-23T23:22:44.973 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-03-23T23:22:45.023 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-03-23T23:22:45.025 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T23:22:45.025 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-03-23T23:22:45.081 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-03-23T23:22:45.081 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-03-23T23:22:45.105 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-03-23T23:22:45.106 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-03-23T23:22:45.126 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-03-23T23:22:45.126 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-03-23T23:22:45.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:22:45.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:22:45.200 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:22:45.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:22:45.229 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:22:45.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:22:45.261 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T23:22:45.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-03-23T23:22:45.344 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-03-23T23:22:45.374 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-03-23T23:22:45.412 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T23:22:45.421 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-23T23:22:45.428 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-23T23:22:45.552 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T23:22:45.568 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T23:22:45.569 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T23:28:37.130 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-03-23T23:28:37.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:28:37.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:28:37.285 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-23T23:28:37.438 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-23T23:28:37.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-23T23:28:37.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:28:37.531 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-23T23:28:37.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-23T23:28:37.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-23T23:28:37.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T23:28:37.691 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-03-23T23:28:37.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-23T23:28:37.762 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T23:28:37.775 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T23:28:37.775 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T23:28:37.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:28:37.783 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T23:28:37.783 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:28:37.785 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T23:28:37.786 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:28:37.816 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T23:28:37.833 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T23:28:37.835 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T23:28:37.853 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T23:28:37.863 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-03-23T23:28:37.877 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-03-23T23:28:37.883 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-03-23T23:28:37.883 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T23:28:37.893 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T23:28:37.897 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-03-23T23:28:37.909 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T23:28:37.914 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T23:28:37.930 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T23:28:37.934 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T23:28:37.953 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:28:37.956 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:28:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-23T23:28:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.957 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.974 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:28:37.976 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:28:37.977 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-03-23T23:28:37.977 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.977 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.977 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:37.977 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:38.035 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-03-23T23:28:38.082 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-03-23T23:28:38.129 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T23:28:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T23:28:38.305 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:28:38.313 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:28:38.313 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-23T23:28:38.314 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:38.314 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:38.314 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:38.314 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:28:38.315 INFO:teuthology.run_tasks:Running task install... 2024-03-23T23:28:38.324 DEBUG:teuthology.task.install:project ceph 2024-03-23T23:28:38.324 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-23T23:28:38.324 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-23T23:28:38.324 INFO:teuthology.task.install:Using flavor: default 2024-03-23T23:28:38.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T23:28:38.330 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T23:28:38.331 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-23T23:28:38.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:28:38.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-23T23:28:38.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:28:38.334 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-23T23:28:38.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:28:38.478 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-23T23:28:38.478 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-23T23:28:38.555 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-23T23:28:38.555 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-23T23:28:38.561 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-23T23:28:38.561 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-23T23:28:38.831 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T23:28:38.832 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:28:38.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T23:28:38.907 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T23:28:38.908 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:28:38.908 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T23:28:38.915 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T23:28:38.915 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:28:38.915 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T23:28:38.935 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T23:28:38.935 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T23:28:38.937 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T23:28:38.938 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T23:28:38.945 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T23:28:38.945 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T23:28:39.007 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T23:28:39.016 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T23:28:39.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T23:28:39.088 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T23:28:39.099 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T23:28:39.154 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-03-23T23:28:39.156 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-03-23T23:28:39.168 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-03-23T23:28:39.169 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-03-23T23:28:39.196 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T23:28:39.287 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-03-23T23:28:39.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-03-23T23:28:39.391 INFO:teuthology.orchestra.run.smithi174.stdout:74 files removed 2024-03-23T23:28:39.407 INFO:teuthology.orchestra.run.smithi149.stdout:74 files removed 2024-03-23T23:28:39.419 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T23:28:39.435 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T23:28:39.644 INFO:teuthology.orchestra.run.smithi099.stdout:74 files removed 2024-03-23T23:28:39.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T23:28:40.579 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 98 kB/s | 83 kB 00:00 2024-03-23T23:28:40.702 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 83 kB/s | 83 kB 00:00 2024-03-23T23:28:41.266 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 78 kB/s | 83 kB 00:01 2024-03-23T23:28:41.380 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-23T23:28:41.494 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-23T23:28:42.239 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-03-23T23:28:42.295 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-03-23T23:28:42.325 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 10 kB/s | 10 kB 00:00 2024-03-23T23:28:42.689 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-23T23:28:42.692 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-23T23:28:43.142 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-03-23T23:28:43.727 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-23T23:28:44.055 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-03-23T23:28:44.125 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-23T23:28:45.004 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-23T23:28:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-23T23:28:45.649 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-03-23T23:28:46.841 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.0 MB 00:01 2024-03-23T23:28:46.872 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-23T23:28:47.760 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-03-23T23:28:49.081 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-23T23:28:53.759 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.1 MB 00:02 2024-03-23T23:28:54.126 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.1 MB 00:01 2024-03-23T23:28:56.071 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-03-23T23:28:56.076 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-23T23:28:57.070 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-23T23:28:57.464 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 879 kB/s | 6.1 MB 00:07 2024-03-23T23:28:57.802 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-23T23:28:59.098 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-23T23:29:02.074 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-03-23T23:29:02.752 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-23T23:29:02.845 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-23T23:29:05.094 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.094 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.107 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T23:29:05.108 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.109 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T23:29:05.110 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.112 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.115 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.116 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T23:29:05.197 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-23T23:29:05.220 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-23T23:29:05.221 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-23T23:29:05.222 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T23:29:05.223 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T23:29:05.224 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T23:29:05.225 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T23:29:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T23:29:05.227 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T23:29:05.228 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T23:29:05.229 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T23:29:05.230 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T23:29:05.231 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T23:29:05.232 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:Install 150 Packages 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 24 Packages 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 224 M 2024-03-23T23:29:05.233 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-23T23:29:05.433 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.434 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.456 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T23:29:05.457 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T23:29:05.458 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T23:29:05.460 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.462 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.466 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T23:29:05.467 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T23:29:05.556 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-23T23:29:05.599 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================ 2024-03-23T23:29:05.599 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-03-23T23:29:05.599 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================ 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-23T23:29:05.600 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T23:29:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T23:29:05.602 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-23T23:29:05.603 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T23:29:05.604 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T23:29:05.605 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T23:29:05.606 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T23:29:05.607 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T23:29:05.608 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T23:29:05.609 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T23:29:05.610 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T23:29:05.611 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T23:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-23T23:29:05.613 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================================ 2024-03-23T23:29:05.614 INFO:teuthology.orchestra.run.smithi099.stdout:Install 150 Packages 2024-03-23T23:29:05.614 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 24 Packages 2024-03-23T23:29:05.614 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:29:05.614 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 224 M 2024-03-23T23:29:05.614 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-03-23T23:29:05.920 INFO:teuthology.orchestra.run.smithi149.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-23T23:29:06.195 INFO:teuthology.orchestra.run.smithi149.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 10 MB/s | 5.4 MB 00:00 2024-03-23T23:29:06.245 INFO:teuthology.orchestra.run.smithi149.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.8 MB/s | 145 kB 00:00 2024-03-23T23:29:06.413 INFO:teuthology.orchestra.run.smithi149.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 14 MB/s | 2.3 MB 00:00 2024-03-23T23:29:06.521 INFO:teuthology.orchestra.run.smithi149.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 14 MB/s | 1.5 MB 00:00 2024-03-23T23:29:06.637 INFO:teuthology.orchestra.run.smithi099.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-23T23:29:06.763 INFO:teuthology.orchestra.run.smithi149.stdout:(6/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 890 kB 00:00 2024-03-23T23:29:06.830 INFO:teuthology.orchestra.run.smithi149.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 15 MB/s | 4.7 MB 00:00 2024-03-23T23:29:06.888 INFO:teuthology.orchestra.run.smithi099.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 3.5 MB/s | 890 kB 00:00 2024-03-23T23:29:06.996 INFO:teuthology.orchestra.run.smithi099.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.3 MB/s | 145 kB 00:00 2024-03-23T23:29:07.065 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-23T23:29:07.413 INFO:teuthology.orchestra.run.smithi099.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 5.6 MB/s | 2.3 MB 00:00 2024-03-23T23:29:07.630 INFO:teuthology.orchestra.run.smithi149.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 12 MB/s | 9.9 MB 00:00 2024-03-23T23:29:07.644 INFO:teuthology.orchestra.run.smithi099.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-03-23T23:29:07.672 INFO:teuthology.orchestra.run.smithi149.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 611 kB/s | 25 kB 00:00 2024-03-23T23:29:08.264 INFO:teuthology.orchestra.run.smithi099.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 7.6 MB/s | 4.7 MB 00:00 2024-03-23T23:29:09.428 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:09.429 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T23:29:09.443 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T23:29:09.444 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T23:29:09.445 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T23:29:09.446 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:09.448 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T23:29:09.451 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T23:29:09.452 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T23:29:09.531 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-23T23:29:09.554 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-23T23:29:09.555 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-23T23:29:09.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T23:29:09.557 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T23:29:09.558 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-23T23:29:09.559 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T23:29:09.560 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T23:29:09.561 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T23:29:09.562 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T23:29:09.563 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T23:29:09.564 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T23:29:09.565 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T23:29:09.566 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================ 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout:Install 150 Packages 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 24 Packages 2024-03-23T23:29:09.567 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:29:09.568 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 224 M 2024-03-23T23:29:09.568 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-23T23:29:09.647 INFO:teuthology.orchestra.run.smithi099.stdout:(7/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-03-23T23:29:10.183 INFO:teuthology.orchestra.run.smithi174.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-23T23:29:10.822 INFO:teuthology.orchestra.run.smithi149.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 4.0 MB/s | 20 MB 00:05 2024-03-23T23:29:11.030 INFO:teuthology.orchestra.run.smithi149.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 1.0 MB/s | 220 kB 00:00 2024-03-23T23:29:11.058 INFO:teuthology.orchestra.run.smithi174.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 890 kB 00:00 2024-03-23T23:29:11.122 INFO:teuthology.orchestra.run.smithi149.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 295 kB/s | 27 kB 00:00 2024-03-23T23:29:11.192 INFO:teuthology.orchestra.run.smithi174.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.1 MB/s | 145 kB 00:00 2024-03-23T23:29:11.198 INFO:teuthology.orchestra.run.smithi149.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 417 kB/s | 31 kB 00:00 2024-03-23T23:29:11.272 INFO:teuthology.orchestra.run.smithi099.stdout:(8/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 5.6 MB/s | 17 MB 00:03 2024-03-23T23:29:11.439 INFO:teuthology.orchestra.run.smithi099.stdout:(9/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 5.5 MB/s | 9.9 MB 00:01 2024-03-23T23:29:11.448 INFO:teuthology.orchestra.run.smithi149.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 2.9 MB/s | 746 kB 00:00 2024-03-23T23:29:11.456 INFO:teuthology.orchestra.run.smithi099.stdout:(10/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el 138 kB/s | 25 kB 00:00 2024-03-23T23:29:11.615 INFO:teuthology.orchestra.run.smithi099.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 1.4 MB/s | 220 kB 00:00 2024-03-23T23:29:11.673 INFO:teuthology.orchestra.run.smithi149.stdout:(15/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 3.4 MB/s | 17 MB 00:04 2024-03-23T23:29:11.698 INFO:teuthology.orchestra.run.smithi099.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 328 kB/s | 27 kB 00:00 2024-03-23T23:29:11.699 INFO:teuthology.orchestra.run.smithi149.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 652 kB/s | 163 kB 00:00 2024-03-23T23:29:11.766 INFO:teuthology.orchestra.run.smithi149.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-23T23:29:11.790 INFO:teuthology.orchestra.run.smithi099.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 344 kB/s | 31 kB 00:00 2024-03-23T23:29:11.974 INFO:teuthology.orchestra.run.smithi149.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 1.8 MB/s | 501 kB 00:00 2024-03-23T23:29:12.066 INFO:teuthology.orchestra.run.smithi149.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 492 kB/s | 45 kB 00:00 2024-03-23T23:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 1.9 MB/s | 746 kB 00:00 2024-03-23T23:29:12.183 INFO:teuthology.orchestra.run.smithi149.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.1 MB/s | 131 kB 00:00 2024-03-23T23:29:12.266 INFO:teuthology.orchestra.run.smithi099.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.6 MB/s | 163 kB 00:00 2024-03-23T23:29:12.283 INFO:teuthology.orchestra.run.smithi149.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.6 MB/s | 162 kB 00:00 2024-03-23T23:29:12.358 INFO:teuthology.orchestra.run.smithi099.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-23T23:29:12.475 INFO:teuthology.orchestra.run.smithi149.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 1.6 MB/s | 321 kB 00:00 2024-03-23T23:29:12.591 INFO:teuthology.orchestra.run.smithi099.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.1 MB/s | 501 kB 00:00 2024-03-23T23:29:12.659 INFO:teuthology.orchestra.run.smithi149.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.6 MB/s | 298 kB 00:00 2024-03-23T23:29:12.776 INFO:teuthology.orchestra.run.smithi149.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 854 kB/s | 99 kB 00:00 2024-03-23T23:29:12.801 INFO:teuthology.orchestra.run.smithi174.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 1.4 MB/s | 2.3 MB 00:01 2024-03-23T23:29:12.876 INFO:teuthology.orchestra.run.smithi149.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 853 kB/s | 85 kB 00:00 2024-03-23T23:29:13.201 INFO:teuthology.orchestra.run.smithi149.stdout:(26/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-03-23T23:29:13.310 INFO:teuthology.orchestra.run.smithi149.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-23T23:29:13.419 INFO:teuthology.orchestra.run.smithi149.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 313 kB/s | 34 kB 00:00 2024-03-23T23:29:13.537 INFO:teuthology.orchestra.run.smithi149.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.2 MB/s | 139 kB 00:00 2024-03-23T23:29:13.942 INFO:teuthology.orchestra.run.smithi174.stdout:(5/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-03-23T23:29:14.101 INFO:teuthology.orchestra.run.smithi174.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-03-23T23:29:14.111 INFO:teuthology.orchestra.run.smithi149.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 3.1 MB/s | 1.8 MB 00:00 2024-03-23T23:29:14.277 INFO:teuthology.orchestra.run.smithi149.stdout:(31/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 2.2 MB/s | 3.1 MB 00:01 2024-03-23T23:29:14.358 INFO:teuthology.orchestra.run.smithi099.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 5.1 MB 00:01 2024-03-23T23:29:14.425 INFO:teuthology.orchestra.run.smithi099.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 690 kB/s | 45 kB 00:00 2024-03-23T23:29:14.492 INFO:teuthology.orchestra.run.smithi099.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 2.0 MB/s | 131 kB 00:00 2024-03-23T23:29:14.584 INFO:teuthology.orchestra.run.smithi099.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.7 MB/s | 162 kB 00:00 2024-03-23T23:29:14.676 INFO:teuthology.orchestra.run.smithi099.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.5 MB/s | 321 kB 00:00 2024-03-23T23:29:14.784 INFO:teuthology.orchestra.run.smithi099.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.7 MB/s | 298 kB 00:00 2024-03-23T23:29:14.819 INFO:teuthology.orchestra.run.smithi149.stdout:(32/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 6.7 MB/s | 48 MB 00:07 2024-03-23T23:29:14.843 INFO:teuthology.orchestra.run.smithi099.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.7 MB/s | 99 kB 00:00 2024-03-23T23:29:14.844 INFO:teuthology.orchestra.run.smithi149.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2145.g3b 438 kB/s | 248 kB 00:00 2024-03-23T23:29:14.902 INFO:teuthology.orchestra.run.smithi099.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.5 MB/s | 85 kB 00:00 2024-03-23T23:29:14.911 INFO:teuthology.orchestra.run.smithi149.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 744 kB/s | 49 kB 00:00 2024-03-23T23:29:14.945 INFO:teuthology.orchestra.run.smithi149.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 165 kB/s | 16 kB 00:00 2024-03-23T23:29:15.029 INFO:teuthology.orchestra.run.smithi149.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.2 MB/s | 264 kB 00:00 2024-03-23T23:29:15.104 INFO:teuthology.orchestra.run.smithi149.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 244 kB/s | 18 kB 00:00 2024-03-23T23:29:15.354 INFO:teuthology.orchestra.run.smithi149.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-23T23:29:15.379 INFO:teuthology.orchestra.run.smithi149.stdout:(39/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 1.7 MB/s | 748 kB 00:00 2024-03-23T23:29:15.421 INFO:teuthology.orchestra.run.smithi149.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-03-23T23:29:15.446 INFO:teuthology.orchestra.run.smithi149.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-23T23:29:15.480 INFO:teuthology.orchestra.run.smithi149.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-23T23:29:15.497 INFO:teuthology.orchestra.run.smithi149.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-23T23:29:15.581 INFO:teuthology.orchestra.run.smithi149.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-23T23:29:15.598 INFO:teuthology.orchestra.run.smithi149.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 172 kB/s | 17 kB 00:00 2024-03-23T23:29:15.615 INFO:teuthology.orchestra.run.smithi149.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-03-23T23:29:15.627 INFO:teuthology.orchestra.run.smithi099.stdout:(26/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 4.3 MB/s | 3.1 MB 00:00 2024-03-23T23:29:15.640 INFO:teuthology.orchestra.run.smithi149.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-23T23:29:15.657 INFO:teuthology.orchestra.run.smithi149.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-23T23:29:15.682 INFO:teuthology.orchestra.run.smithi149.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-23T23:29:15.711 INFO:teuthology.orchestra.run.smithi149.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-23T23:29:15.728 INFO:teuthology.orchestra.run.smithi149.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.0 MB/s | 46 kB 00:00 2024-03-23T23:29:15.753 INFO:teuthology.orchestra.run.smithi149.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.7 MB/s | 282 kB 00:00 2024-03-23T23:29:15.778 INFO:teuthology.orchestra.run.smithi149.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T23:29:15.804 INFO:teuthology.orchestra.run.smithi149.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-23T23:29:15.829 INFO:teuthology.orchestra.run.smithi149.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-23T23:29:15.902 INFO:teuthology.orchestra.run.smithi099.stdout:(27/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 2.1 MB/s | 20 MB 00:09 2024-03-23T23:29:15.936 INFO:teuthology.orchestra.run.smithi099.stdout:(28/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 554 kB/s | 170 kB 00:00 2024-03-23T23:29:15.946 INFO:teuthology.orchestra.run.smithi149.stdout:(56/174): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-03-23T23:29:15.978 INFO:teuthology.orchestra.run.smithi099.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 456 kB/s | 34 kB 00:00 2024-03-23T23:29:16.012 INFO:teuthology.orchestra.run.smithi099.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.8 MB/s | 139 kB 00:00 2024-03-23T23:29:16.022 INFO:teuthology.orchestra.run.smithi149.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-03-23T23:29:16.064 INFO:teuthology.orchestra.run.smithi149.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 6.6 MB/s | 1.5 MB 00:00 2024-03-23T23:29:16.097 INFO:teuthology.orchestra.run.smithi149.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.4 MB/s | 811 kB 00:00 2024-03-23T23:29:16.122 INFO:teuthology.orchestra.run.smithi149.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-23T23:29:16.173 INFO:teuthology.orchestra.run.smithi149.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-03-23T23:29:16.234 INFO:teuthology.orchestra.run.smithi174.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-03-23T23:29:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-23T23:29:16.315 INFO:teuthology.orchestra.run.smithi149.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 406 kB/s | 78 kB 00:00 2024-03-23T23:29:16.398 INFO:teuthology.orchestra.run.smithi149.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-23T23:29:16.424 INFO:teuthology.orchestra.run.smithi149.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-23T23:29:16.448 INFO:teuthology.orchestra.run.smithi149.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-23T23:29:16.493 INFO:teuthology.orchestra.run.smithi149.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-03-23T23:29:16.524 INFO:teuthology.orchestra.run.smithi149.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-03-23T23:29:16.562 INFO:teuthology.orchestra.run.smithi099.stdout:(31/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 9.3 MB/s | 48 MB 00:05 2024-03-23T23:29:16.600 INFO:teuthology.orchestra.run.smithi149.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-23T23:29:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-23T23:29:16.650 INFO:teuthology.orchestra.run.smithi149.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-23T23:29:16.667 INFO:teuthology.orchestra.run.smithi149.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-23T23:29:16.671 INFO:teuthology.orchestra.run.smithi099.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 2.6 MB/s | 1.8 MB 00:00 2024-03-23T23:29:16.692 INFO:teuthology.orchestra.run.smithi149.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 990 kB/s | 41 kB 00:00 2024-03-23T23:29:16.704 INFO:teuthology.orchestra.run.smithi099.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2145.g3b 1.7 MB/s | 248 kB 00:00 2024-03-23T23:29:16.717 INFO:teuthology.orchestra.run.smithi149.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-23T23:29:16.729 INFO:teuthology.orchestra.run.smithi099.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 855 kB/s | 49 kB 00:00 2024-03-23T23:29:16.743 INFO:teuthology.orchestra.run.smithi149.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-23T23:29:16.754 INFO:teuthology.orchestra.run.smithi099.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 332 kB/s | 16 kB 00:00 2024-03-23T23:29:16.768 INFO:teuthology.orchestra.run.smithi149.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 953 kB/s | 47 kB 00:00 2024-03-23T23:29:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-23T23:29:16.818 INFO:teuthology.orchestra.run.smithi149.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-03-23T23:29:16.855 INFO:teuthology.orchestra.run.smithi099.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.1 MB/s | 264 kB 00:00 2024-03-23T23:29:16.923 INFO:teuthology.orchestra.run.smithi099.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 4.4 MB/s | 748 kB 00:00 2024-03-23T23:29:16.956 INFO:teuthology.orchestra.run.smithi099.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 183 kB/s | 18 kB 00:00 2024-03-23T23:29:16.985 INFO:teuthology.orchestra.run.smithi149.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 237 kB/s | 39 kB 00:00 2024-03-23T23:29:17.144 INFO:teuthology.orchestra.run.smithi149.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-23T23:29:17.169 INFO:teuthology.orchestra.run.smithi149.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 118 kB/s | 41 kB 00:00 2024-03-23T23:29:17.181 INFO:teuthology.orchestra.run.smithi099.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-03-23T23:29:17.194 INFO:teuthology.orchestra.run.smithi149.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-23T23:29:17.220 INFO:teuthology.orchestra.run.smithi149.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-23T23:29:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2024-03-23T23:29:17.245 INFO:teuthology.orchestra.run.smithi149.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 840 kB/s | 63 kB 00:00 2024-03-23T23:29:17.248 INFO:teuthology.orchestra.run.smithi099.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.7 MB/s | 106 kB 00:00 2024-03-23T23:29:17.270 INFO:teuthology.orchestra.run.smithi149.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-03-23T23:29:17.282 INFO:teuthology.orchestra.run.smithi099.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-23T23:29:17.307 INFO:teuthology.orchestra.run.smithi099.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 573 kB/s | 32 kB 00:00 2024-03-23T23:29:17.372 INFO:teuthology.orchestra.run.smithi149.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 241 kB 00:00 2024-03-23T23:29:17.399 INFO:teuthology.orchestra.run.smithi099.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-23T23:29:17.418 INFO:teuthology.orchestra.run.smithi174.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 2.7 MB/s | 20 MB 00:07 2024-03-23T23:29:17.424 INFO:teuthology.orchestra.run.smithi099.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 17 kB 00:00 2024-03-23T23:29:17.451 INFO:teuthology.orchestra.run.smithi174.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 762 kB/s | 25 kB 00:00 2024-03-23T23:29:17.454 INFO:teuthology.orchestra.run.smithi149.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 377 kB/s | 79 kB 00:00 2024-03-23T23:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 248 kB/s | 10 kB 00:00 2024-03-23T23:29:17.488 INFO:teuthology.orchestra.run.smithi149.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.1 MB 00:00 2024-03-23T23:29:17.513 INFO:teuthology.orchestra.run.smithi149.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 740 kB/s | 103 kB 00:00 2024-03-23T23:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-23T23:29:17.542 INFO:teuthology.orchestra.run.smithi099.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 44 kB 00:00 2024-03-23T23:29:17.546 INFO:teuthology.orchestra.run.smithi149.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-23T23:29:17.572 INFO:teuthology.orchestra.run.smithi149.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-23T23:29:17.583 INFO:teuthology.orchestra.run.smithi099.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-23T23:29:17.634 INFO:teuthology.orchestra.run.smithi099.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-23T23:29:17.659 INFO:teuthology.orchestra.run.smithi099.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 666 kB/s | 46 kB 00:00 2024-03-23T23:29:17.693 INFO:teuthology.orchestra.run.smithi099.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-23T23:29:17.697 INFO:teuthology.orchestra.run.smithi149.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-23T23:29:17.722 INFO:teuthology.orchestra.run.smithi149.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-23T23:29:17.726 INFO:teuthology.orchestra.run.smithi099.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-23T23:29:17.747 INFO:teuthology.orchestra.run.smithi149.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-23T23:29:17.760 INFO:teuthology.orchestra.run.smithi099.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-23T23:29:17.781 INFO:teuthology.orchestra.run.smithi149.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-03-23T23:29:17.785 INFO:teuthology.orchestra.run.smithi099.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 34 kB 00:00 2024-03-23T23:29:17.914 INFO:teuthology.orchestra.run.smithi149.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 112 kB/s | 21 kB 00:00 2024-03-23T23:29:17.927 INFO:teuthology.orchestra.run.smithi099.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2024-03-23T23:29:17.939 INFO:teuthology.orchestra.run.smithi149.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 133 kB/s | 25 kB 00:00 2024-03-23T23:29:17.978 INFO:teuthology.orchestra.run.smithi099.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 8.6 MB/s | 1.5 MB 00:00 2024-03-23T23:29:18.018 INFO:teuthology.orchestra.run.smithi174.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 4.3 MB/s | 17 MB 00:03 2024-03-23T23:29:18.028 INFO:teuthology.orchestra.run.smithi099.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 7.9 MB/s | 811 kB 00:00 2024-03-23T23:29:18.053 INFO:teuthology.orchestra.run.smithi099.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2024-03-23T23:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 3.3 MB/s | 220 kB 00:00 2024-03-23T23:29:18.098 INFO:teuthology.orchestra.run.smithi149.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-23T23:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 813 kB/s | 27 kB 00:00 2024-03-23T23:29:18.123 INFO:teuthology.orchestra.run.smithi149.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-23T23:29:18.152 INFO:teuthology.orchestra.run.smithi174.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 943 kB/s | 31 kB 00:00 2024-03-23T23:29:18.179 INFO:teuthology.orchestra.run.smithi099.stdout:(60/174): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-03-23T23:29:18.190 INFO:teuthology.orchestra.run.smithi149.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-23T23:29:18.207 INFO:teuthology.orchestra.run.smithi149.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 417 kB/s | 35 kB 00:00 2024-03-23T23:29:18.229 INFO:teuthology.orchestra.run.smithi099.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-23T23:29:18.274 INFO:teuthology.orchestra.run.smithi149.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 532 kB/s | 44 kB 00:00 2024-03-23T23:29:18.285 INFO:teuthology.orchestra.run.smithi174.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 5.5 MB/s | 746 kB 00:00 2024-03-23T23:29:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-23T23:29:18.336 INFO:teuthology.orchestra.run.smithi174.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 3.2 MB/s | 163 kB 00:00 2024-03-23T23:29:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-03-23T23:29:18.386 INFO:teuthology.orchestra.run.smithi174.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.5 MB/s | 127 kB 00:00 2024-03-23T23:29:18.421 INFO:teuthology.orchestra.run.smithi099.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-23T23:29:18.455 INFO:teuthology.orchestra.run.smithi099.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 292 kB/s | 78 kB 00:00 2024-03-23T23:29:18.487 INFO:teuthology.orchestra.run.smithi174.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.9 MB/s | 501 kB 00:00 2024-03-23T23:29:18.542 INFO:teuthology.orchestra.run.smithi149.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-03-23T23:29:18.580 INFO:teuthology.orchestra.run.smithi099.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-03-23T23:29:18.605 INFO:teuthology.orchestra.run.smithi099.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 142 kB/s | 24 kB 00:00 2024-03-23T23:29:18.622 INFO:teuthology.orchestra.run.smithi099.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 118 kB/s | 18 kB 00:00 2024-03-23T23:29:18.672 INFO:teuthology.orchestra.run.smithi099.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 483 kB/s | 32 kB 00:00 2024-03-23T23:29:18.700 INFO:teuthology.orchestra.run.smithi149.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.5 MB/s | 4.1 MB 00:00 2024-03-23T23:29:18.714 INFO:teuthology.orchestra.run.smithi099.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-23T23:29:18.734 INFO:teuthology.orchestra.run.smithi149.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-23T23:29:18.839 INFO:teuthology.orchestra.run.smithi099.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-03-23T23:29:18.845 INFO:teuthology.orchestra.run.smithi174.stdout:(18/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 3.8 MB/s | 9.9 MB 00:02 2024-03-23T23:29:18.851 INFO:teuthology.orchestra.run.smithi149.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 998 kB/s | 309 kB 00:00 2024-03-23T23:29:18.864 INFO:teuthology.orchestra.run.smithi099.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-23T23:29:18.869 INFO:teuthology.orchestra.run.smithi149.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 116 kB/s | 19 kB 00:00 2024-03-23T23:29:18.890 INFO:teuthology.orchestra.run.smithi099.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-23T23:29:18.904 INFO:teuthology.orchestra.run.smithi174.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 779 kB/s | 45 kB 00:00 2024-03-23T23:29:18.910 INFO:teuthology.orchestra.run.smithi149.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-03-23T23:29:18.923 INFO:teuthology.orchestra.run.smithi099.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2024-03-23T23:29:18.949 INFO:teuthology.orchestra.run.smithi099.stdout:(73/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-23T23:29:18.961 INFO:teuthology.orchestra.run.smithi149.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-03-23T23:29:18.970 INFO:teuthology.orchestra.run.smithi174.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.9 MB/s | 131 kB 00:00 2024-03-23T23:29:18.974 INFO:teuthology.orchestra.run.smithi099.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-23T23:29:18.991 INFO:teuthology.orchestra.run.smithi099.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-23T23:29:18.994 INFO:teuthology.orchestra.run.smithi149.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-03-23T23:29:19.016 INFO:teuthology.orchestra.run.smithi099.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 47 kB 00:00 2024-03-23T23:29:19.037 INFO:teuthology.orchestra.run.smithi174.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.4 MB/s | 162 kB 00:00 2024-03-23T23:29:19.049 INFO:teuthology.orchestra.run.smithi099.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-23T23:29:19.070 INFO:teuthology.orchestra.run.smithi149.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-23T23:29:19.075 INFO:teuthology.orchestra.run.smithi099.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 518 kB/s | 43 kB 00:00 2024-03-23T23:29:19.154 INFO:teuthology.orchestra.run.smithi174.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.7 MB/s | 321 kB 00:00 2024-03-23T23:29:19.195 INFO:teuthology.orchestra.run.smithi149.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-03-23T23:29:19.220 INFO:teuthology.orchestra.run.smithi149.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-03-23T23:29:19.246 INFO:teuthology.orchestra.run.smithi174.stdout:(23/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 6.8 MB/s | 5.1 MB 00:00 2024-03-23T23:29:19.250 INFO:teuthology.orchestra.run.smithi099.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 257 kB/s | 41 kB 00:00 2024-03-23T23:29:19.254 INFO:teuthology.orchestra.run.smithi149.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-03-23T23:29:19.275 INFO:teuthology.orchestra.run.smithi099.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 209 kB/s | 39 kB 00:00 2024-03-23T23:29:19.279 INFO:teuthology.orchestra.run.smithi149.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-03-23T23:29:19.297 INFO:teuthology.orchestra.run.smithi174.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 2.0 MB/s | 99 kB 00:00 2024-03-23T23:29:19.313 INFO:teuthology.orchestra.run.smithi149.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-23T23:29:19.322 INFO:teuthology.orchestra.run.smithi174.stdout:(25/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.7 MB/s | 298 kB 00:00 2024-03-23T23:29:19.346 INFO:teuthology.orchestra.run.smithi149.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-23T23:29:19.347 INFO:teuthology.orchestra.run.smithi174.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-23T23:29:19.397 INFO:teuthology.orchestra.run.smithi174.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-23T23:29:19.421 INFO:teuthology.orchestra.run.smithi149.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-03-23T23:29:19.448 INFO:teuthology.orchestra.run.smithi174.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 678 kB/s | 34 kB 00:00 2024-03-23T23:29:19.455 INFO:teuthology.orchestra.run.smithi149.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-03-23T23:29:19.481 INFO:teuthology.orchestra.run.smithi149.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-03-23T23:29:19.498 INFO:teuthology.orchestra.run.smithi174.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.7 MB/s | 139 kB 00:00 2024-03-23T23:29:19.505 INFO:teuthology.orchestra.run.smithi149.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-03-23T23:29:19.531 INFO:teuthology.orchestra.run.smithi149.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-23T23:29:19.543 INFO:teuthology.orchestra.run.smithi099.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-03-23T23:29:19.567 INFO:teuthology.orchestra.run.smithi099.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 948 kB/s | 300 kB 00:00 2024-03-23T23:29:19.573 INFO:teuthology.orchestra.run.smithi149.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-23T23:29:19.593 INFO:teuthology.orchestra.run.smithi099.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 184 kB/s | 56 kB 00:00 2024-03-23T23:29:19.598 INFO:teuthology.orchestra.run.smithi149.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-23T23:29:19.618 INFO:teuthology.orchestra.run.smithi099.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-23T23:29:19.642 INFO:teuthology.orchestra.run.smithi149.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-23T23:29:19.643 INFO:teuthology.orchestra.run.smithi099.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 600 kB/s | 44 kB 00:00 2024-03-23T23:29:19.665 INFO:teuthology.orchestra.run.smithi149.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-03-23T23:29:19.669 INFO:teuthology.orchestra.run.smithi099.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-23T23:29:19.707 INFO:teuthology.orchestra.run.smithi149.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-03-23T23:29:19.719 INFO:teuthology.orchestra.run.smithi099.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-23T23:29:19.731 INFO:teuthology.orchestra.run.smithi174.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 7.7 MB/s | 1.8 MB 00:00 2024-03-23T23:29:19.732 INFO:teuthology.orchestra.run.smithi149.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-23T23:29:19.761 INFO:teuthology.orchestra.run.smithi099.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.1 MB 00:00 2024-03-23T23:29:19.799 INFO:teuthology.orchestra.run.smithi149.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-03-23T23:29:19.803 INFO:teuthology.orchestra.run.smithi099.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 845 kB/s | 103 kB 00:00 2024-03-23T23:29:19.836 INFO:teuthology.orchestra.run.smithi099.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-23T23:29:19.862 INFO:teuthology.orchestra.run.smithi099.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2024-03-23T23:29:19.874 INFO:teuthology.orchestra.run.smithi149.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-03-23T23:29:19.900 INFO:teuthology.orchestra.run.smithi149.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-03-23T23:29:19.933 INFO:teuthology.orchestra.run.smithi149.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-03-23T23:29:19.959 INFO:teuthology.orchestra.run.smithi149.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-03-23T23:29:19.984 INFO:teuthology.orchestra.run.smithi149.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-23T23:29:20.007 INFO:teuthology.orchestra.run.smithi174.stdout:(31/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 4.5 MB/s | 3.1 MB 00:00 2024-03-23T23:29:20.009 INFO:teuthology.orchestra.run.smithi149.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-03-23T23:29:20.035 INFO:teuthology.orchestra.run.smithi149.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-23T23:29:20.060 INFO:teuthology.orchestra.run.smithi149.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-03-23T23:29:20.085 INFO:teuthology.orchestra.run.smithi149.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 405 kB/s | 31 kB 00:00 2024-03-23T23:29:20.107 INFO:teuthology.orchestra.run.smithi174.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.4 MB/s | 248 kB 00:00 2024-03-23T23:29:20.110 INFO:teuthology.orchestra.run.smithi149.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-23T23:29:20.136 INFO:teuthology.orchestra.run.smithi149.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-23T23:29:20.161 INFO:teuthology.orchestra.run.smithi149.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-23T23:29:20.171 INFO:teuthology.orchestra.run.smithi099.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 514 kB/s | 187 kB 00:00 2024-03-23T23:29:20.174 INFO:teuthology.orchestra.run.smithi174.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 741 kB/s | 49 kB 00:00 2024-03-23T23:29:20.186 INFO:teuthology.orchestra.run.smithi149.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-23T23:29:20.211 INFO:teuthology.orchestra.run.smithi149.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-23T23:29:20.212 INFO:teuthology.orchestra.run.smithi099.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 560 kB/s | 195 kB 00:00 2024-03-23T23:29:20.232 INFO:teuthology.orchestra.run.smithi174.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 283 kB/s | 16 kB 00:00 2024-03-23T23:29:20.237 INFO:teuthology.orchestra.run.smithi149.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-23T23:29:20.279 INFO:teuthology.orchestra.run.smithi149.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-03-23T23:29:20.304 INFO:teuthology.orchestra.run.smithi149.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-23T23:29:20.324 INFO:teuthology.orchestra.run.smithi174.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.8 MB/s | 264 kB 00:00 2024-03-23T23:29:20.329 INFO:teuthology.orchestra.run.smithi149.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-23T23:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-23T23:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 354 kB/s | 25 kB 00:00 2024-03-23T23:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 145 kB/s | 21 kB 00:00 2024-03-23T23:29:20.464 INFO:teuthology.orchestra.run.smithi099.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 428 kB/s | 125 kB 00:00 2024-03-23T23:29:20.479 INFO:teuthology.orchestra.run.smithi149.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-23T23:29:20.508 INFO:teuthology.orchestra.run.smithi174.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 4.0 MB/s | 748 kB 00:00 2024-03-23T23:29:20.521 INFO:teuthology.orchestra.run.smithi149.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-23T23:29:20.539 INFO:teuthology.orchestra.run.smithi099.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-23T23:29:20.573 INFO:teuthology.orchestra.run.smithi099.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-23T23:29:20.575 INFO:teuthology.orchestra.run.smithi174.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 274 kB/s | 18 kB 00:00 2024-03-23T23:29:20.623 INFO:teuthology.orchestra.run.smithi099.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 711 kB/s | 35 kB 00:00 2024-03-23T23:29:20.666 INFO:teuthology.orchestra.run.smithi099.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-23T23:29:20.692 INFO:teuthology.orchestra.run.smithi174.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 7.7 MB/s | 7.4 MB 00:00 2024-03-23T23:29:20.698 INFO:teuthology.orchestra.run.smithi099.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 598 kB/s | 44 kB 00:00 2024-03-23T23:29:20.705 INFO:teuthology.orchestra.run.smithi149.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-23T23:29:20.757 INFO:teuthology.orchestra.run.smithi099.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-23T23:29:20.784 INFO:teuthology.orchestra.run.smithi174.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-23T23:29:20.797 INFO:teuthology.orchestra.run.smithi149.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-23T23:29:20.817 INFO:teuthology.orchestra.run.smithi174.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 920 kB 00:00 2024-03-23T23:29:20.839 INFO:teuthology.orchestra.run.smithi149.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-23T23:29:20.842 INFO:teuthology.orchestra.run.smithi174.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-23T23:29:20.866 INFO:teuthology.orchestra.run.smithi099.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.0 MB/s | 4.1 MB 00:00 2024-03-23T23:29:20.869 INFO:teuthology.orchestra.run.smithi174.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T23:29:20.872 INFO:teuthology.orchestra.run.smithi149.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 7.3 MB/s | 248 kB 00:00 2024-03-23T23:29:20.883 INFO:teuthology.orchestra.run.smithi174.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 801 kB/s | 32 kB 00:00 2024-03-23T23:29:20.906 INFO:teuthology.orchestra.run.smithi149.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2024-03-23T23:29:20.950 INFO:teuthology.orchestra.run.smithi174.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 37 MB/s | 3.0 MB 00:00 2024-03-23T23:29:20.956 INFO:teuthology.orchestra.run.smithi149.stdout:(157/174): librados2-19.0.0-2145.g3be9a5c9.el9. 5.2 MB/s | 3.4 MB 00:00 2024-03-23T23:29:20.967 INFO:teuthology.orchestra.run.smithi174.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 207 kB/s | 17 kB 00:00 2024-03-23T23:29:20.973 INFO:teuthology.orchestra.run.smithi149.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 545 kB/s | 36 kB 00:00 2024-03-23T23:29:20.984 INFO:teuthology.orchestra.run.smithi174.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-03-23T23:29:21.001 INFO:teuthology.orchestra.run.smithi174.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.6 MB/s | 159 kB 00:00 2024-03-23T23:29:21.015 INFO:teuthology.orchestra.run.smithi149.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-23T23:29:21.018 INFO:teuthology.orchestra.run.smithi174.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-23T23:29:21.043 INFO:teuthology.orchestra.run.smithi174.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 661 kB 00:00 2024-03-23T23:29:21.049 INFO:teuthology.orchestra.run.smithi149.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 697 kB/s | 23 kB 00:00 2024-03-23T23:29:21.069 INFO:teuthology.orchestra.run.smithi174.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-23T23:29:21.074 INFO:teuthology.orchestra.run.smithi149.stdout:(161/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-03-23T23:29:21.094 INFO:teuthology.orchestra.run.smithi174.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2024-03-23T23:29:21.099 INFO:teuthology.orchestra.run.smithi149.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-23T23:29:21.116 INFO:teuthology.orchestra.run.smithi149.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-03-23T23:29:21.119 INFO:teuthology.orchestra.run.smithi174.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-23T23:29:21.142 INFO:teuthology.orchestra.run.smithi149.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-23T23:29:21.144 INFO:teuthology.orchestra.run.smithi174.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T23:29:21.168 INFO:teuthology.orchestra.run.smithi149.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-23T23:29:21.170 INFO:teuthology.orchestra.run.smithi174.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-23T23:29:21.187 INFO:teuthology.orchestra.run.smithi174.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2024-03-23T23:29:21.192 INFO:teuthology.orchestra.run.smithi149.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-23T23:29:21.251 INFO:teuthology.orchestra.run.smithi149.stdout:(167/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-03-23T23:29:21.293 INFO:teuthology.orchestra.run.smithi149.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-23T23:29:21.295 INFO:teuthology.orchestra.run.smithi174.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-23T23:29:21.338 INFO:teuthology.orchestra.run.smithi174.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-23T23:29:21.343 INFO:teuthology.orchestra.run.smithi149.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-23T23:29:21.369 INFO:teuthology.orchestra.run.smithi149.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-23T23:29:21.371 INFO:teuthology.orchestra.run.smithi174.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-23T23:29:21.394 INFO:teuthology.orchestra.run.smithi149.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-03-23T23:29:21.396 INFO:teuthology.orchestra.run.smithi174.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-23T23:29:21.436 INFO:teuthology.orchestra.run.smithi149.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-23T23:29:21.449 INFO:teuthology.orchestra.run.smithi099.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-03-23T23:29:21.486 INFO:teuthology.orchestra.run.smithi149.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.7 MB/s | 2.3 MB 00:00 2024-03-23T23:29:21.505 INFO:teuthology.orchestra.run.smithi174.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 43 MB/s | 5.7 MB 00:00 2024-03-23T23:29:21.536 INFO:teuthology.orchestra.run.smithi149.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 524 kB/s | 74 kB 00:00 2024-03-23T23:29:21.538 INFO:teuthology.orchestra.run.smithi174.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-03-23T23:29:21.540 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-23T23:29:21.540 INFO:teuthology.orchestra.run.smithi149.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-23T23:29:21.563 INFO:teuthology.orchestra.run.smithi174.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-23T23:29:21.681 INFO:teuthology.orchestra.run.smithi174.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-23T23:29:21.706 INFO:teuthology.orchestra.run.smithi174.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-23T23:29:21.732 INFO:teuthology.orchestra.run.smithi174.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 359 kB/s | 18 kB 00:00 2024-03-23T23:29:21.756 INFO:teuthology.orchestra.run.smithi174.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-23T23:29:21.782 INFO:teuthology.orchestra.run.smithi174.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-23T23:29:21.874 INFO:teuthology.orchestra.run.smithi174.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-03-23T23:29:21.900 INFO:teuthology.orchestra.run.smithi174.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2024-03-23T23:29:21.924 INFO:teuthology.orchestra.run.smithi174.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-23T23:29:21.949 INFO:teuthology.orchestra.run.smithi174.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-23T23:29:21.975 INFO:teuthology.orchestra.run.smithi174.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-23T23:29:22.000 INFO:teuthology.orchestra.run.smithi174.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-23T23:29:22.020 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-23T23:29:22.025 INFO:teuthology.orchestra.run.smithi174.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-23T23:29:22.050 INFO:teuthology.orchestra.run.smithi174.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-23T23:29:22.076 INFO:teuthology.orchestra.run.smithi174.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-03-23T23:29:22.094 INFO:teuthology.orchestra.run.smithi174.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-23T23:29:22.149 INFO:teuthology.orchestra.run.smithi099.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 427 kB/s | 548 kB 00:01 2024-03-23T23:29:22.168 INFO:teuthology.orchestra.run.smithi174.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 522 kB/s | 39 kB 00:00 2024-03-23T23:29:22.258 INFO:teuthology.orchestra.run.smithi099.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 181 kB/s | 19 kB 00:00 2024-03-23T23:29:22.295 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-23T23:29:22.296 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-23T23:29:22.383 INFO:teuthology.orchestra.run.smithi099.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 331 kB/s | 309 kB 00:00 2024-03-23T23:29:22.435 INFO:teuthology.orchestra.run.smithi174.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-23T23:29:22.452 INFO:teuthology.orchestra.run.smithi174.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 146 kB/s | 41 kB 00:00 2024-03-23T23:29:22.477 INFO:teuthology.orchestra.run.smithi174.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-23T23:29:22.483 INFO:teuthology.orchestra.run.smithi099.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-03-23T23:29:22.494 INFO:teuthology.orchestra.run.smithi174.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-23T23:29:22.511 INFO:teuthology.orchestra.run.smithi174.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-23T23:29:22.529 INFO:teuthology.orchestra.run.smithi174.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-23T23:29:22.553 INFO:teuthology.orchestra.run.smithi174.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-23T23:29:22.579 INFO:teuthology.orchestra.run.smithi174.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-23T23:29:22.592 INFO:teuthology.orchestra.run.smithi099.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 451 kB/s | 49 kB 00:00 2024-03-23T23:29:22.612 INFO:teuthology.orchestra.run.smithi174.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-23T23:29:22.637 INFO:teuthology.orchestra.run.smithi174.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-23T23:29:22.642 INFO:teuthology.orchestra.run.smithi099.stdout:(111/174): protobuf-compiler-3.14.0-13.el9.x86_ 458 kB/s | 863 kB 00:01 2024-03-23T23:29:22.663 INFO:teuthology.orchestra.run.smithi174.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-23T23:29:22.688 INFO:teuthology.orchestra.run.smithi174.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-23T23:29:22.701 INFO:teuthology.orchestra.run.smithi099.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-03-23T23:29:23.122 INFO:teuthology.orchestra.run.smithi174.stdout:(91/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 8.4 MB/s | 48 MB 00:05 2024-03-23T23:29:23.188 INFO:teuthology.orchestra.run.smithi174.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 356 kB/s | 187 kB 00:00 2024-03-23T23:29:23.297 INFO:teuthology.orchestra.run.smithi174.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-03-23T23:29:23.322 INFO:teuthology.orchestra.run.smithi174.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 935 kB/s | 125 kB 00:00 2024-03-23T23:29:23.348 INFO:teuthology.orchestra.run.smithi174.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 864 kB/s | 195 kB 00:00 2024-03-23T23:29:23.351 INFO:teuthology.orchestra.run.smithi099.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 213 kB/s | 151 kB 00:00 2024-03-23T23:29:23.582 INFO:teuthology.orchestra.run.smithi174.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 76 kB/s | 21 kB 00:00 2024-03-23T23:29:23.607 INFO:teuthology.orchestra.run.smithi174.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 89 kB/s | 25 kB 00:00 2024-03-23T23:29:23.793 INFO:teuthology.orchestra.run.smithi099.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 768 kB/s | 838 kB 00:01 2024-03-23T23:29:23.910 INFO:teuthology.orchestra.run.smithi099.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 373 kB/s | 43 kB 00:00 2024-03-23T23:29:23.949 INFO:teuthology.orchestra.run.smithi174.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 719 kB/s | 245 kB 00:00 2024-03-23T23:29:23.974 INFO:teuthology.orchestra.run.smithi174.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 676 kB/s | 248 kB 00:00 2024-03-23T23:29:24.019 INFO:teuthology.orchestra.run.smithi099.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-03-23T23:29:24.023 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-23T23:29:24.023 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-23T23:29:24.083 INFO:teuthology.orchestra.run.smithi174.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 322 kB/s | 35 kB 00:00 2024-03-23T23:29:24.128 INFO:teuthology.orchestra.run.smithi099.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-03-23T23:29:24.133 INFO:teuthology.orchestra.run.smithi174.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-23T23:29:24.161 INFO:teuthology.orchestra.run.smithi099.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 677 kB/s | 548 kB 00:00 2024-03-23T23:29:24.192 INFO:teuthology.orchestra.run.smithi174.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 410 kB/s | 44 kB 00:00 2024-03-23T23:29:24.233 INFO:teuthology.orchestra.run.smithi174.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 936 kB/s | 94 kB 00:00 2024-03-23T23:29:24.258 INFO:teuthology.orchestra.run.smithi174.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-03-23T23:29:24.392 INFO:teuthology.orchestra.run.smithi174.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-03-23T23:29:24.420 INFO:teuthology.orchestra.run.smithi099.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 590 kB/s | 172 kB 00:00 2024-03-23T23:29:24.612 INFO:teuthology.orchestra.run.smithi099.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 797 kB/s | 358 kB 00:00 2024-03-23T23:29:24.718 INFO:teuthology.orchestra.run.smithi174.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-03-23T23:29:24.720 INFO:teuthology.orchestra.run.smithi099.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 328 kB/s | 35 kB 00:00 2024-03-23T23:29:24.743 INFO:teuthology.orchestra.run.smithi174.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 885 kB/s | 309 kB 00:00 2024-03-23T23:29:24.785 INFO:teuthology.orchestra.run.smithi174.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-03-23T23:29:24.854 INFO:teuthology.orchestra.run.smithi099.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 700 kB/s | 303 kB 00:00 2024-03-23T23:29:24.918 INFO:teuthology.orchestra.run.smithi174.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-03-23T23:29:24.977 INFO:teuthology.orchestra.run.smithi174.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-03-23T23:29:25.002 INFO:teuthology.orchestra.run.smithi174.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-03-23T23:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 695 kB/s | 203 kB 00:00 2024-03-23T23:29:25.036 INFO:teuthology.orchestra.run.smithi174.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-23T23:29:25.069 INFO:teuthology.orchestra.run.smithi174.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-23T23:29:25.111 INFO:teuthology.orchestra.run.smithi174.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-03-23T23:29:25.145 INFO:teuthology.orchestra.run.smithi174.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-03-23T23:29:25.170 INFO:teuthology.orchestra.run.smithi174.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-03-23T23:29:25.204 INFO:teuthology.orchestra.run.smithi174.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-23T23:29:25.229 INFO:teuthology.orchestra.run.smithi174.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-23T23:29:25.271 INFO:teuthology.orchestra.run.smithi174.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-23T23:29:25.288 INFO:teuthology.orchestra.run.smithi099.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 526 kB/s | 144 kB 00:00 2024-03-23T23:29:25.330 INFO:teuthology.orchestra.run.smithi174.stdout:(120/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.1 MB/s | 4.1 MB 00:01 2024-03-23T23:29:25.355 INFO:teuthology.orchestra.run.smithi174.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-23T23:29:25.380 INFO:teuthology.orchestra.run.smithi174.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-23T23:29:25.397 INFO:teuthology.orchestra.run.smithi174.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 539 kB/s | 35 kB 00:00 2024-03-23T23:29:25.405 INFO:teuthology.orchestra.run.smithi099.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-23T23:29:25.455 INFO:teuthology.orchestra.run.smithi174.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-23T23:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-03-23T23:29:25.564 INFO:teuthology.orchestra.run.smithi174.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-23T23:29:25.581 INFO:teuthology.orchestra.run.smithi174.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 786 kB/s | 144 kB 00:00 2024-03-23T23:29:25.598 INFO:teuthology.orchestra.run.smithi174.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 76 kB/s | 11 kB 00:00 2024-03-23T23:29:25.622 INFO:teuthology.orchestra.run.smithi099.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-03-23T23:29:25.623 INFO:teuthology.orchestra.run.smithi174.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-03-23T23:29:25.657 INFO:teuthology.orchestra.run.smithi174.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-03-23T23:29:25.673 INFO:teuthology.orchestra.run.smithi174.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-03-23T23:29:25.690 INFO:teuthology.orchestra.run.smithi174.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-23T23:29:25.731 INFO:teuthology.orchestra.run.smithi099.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-03-23T23:29:25.765 INFO:teuthology.orchestra.run.smithi174.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-23T23:29:25.782 INFO:teuthology.orchestra.run.smithi174.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-23T23:29:25.807 INFO:teuthology.orchestra.run.smithi174.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-03-23T23:29:25.824 INFO:teuthology.orchestra.run.smithi174.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-03-23T23:29:25.839 INFO:teuthology.orchestra.run.smithi099.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 183 kB/s | 20 kB 00:00 2024-03-23T23:29:25.900 INFO:teuthology.orchestra.run.smithi174.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-23T23:29:25.916 INFO:teuthology.orchestra.run.smithi174.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-23T23:29:25.942 INFO:teuthology.orchestra.run.smithi174.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-03-23T23:29:25.984 INFO:teuthology.orchestra.run.smithi174.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-03-23T23:29:26.009 INFO:teuthology.orchestra.run.smithi174.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-03-23T23:29:26.043 INFO:teuthology.orchestra.run.smithi174.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-23T23:29:26.068 INFO:teuthology.orchestra.run.smithi174.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 703 kB/s | 59 kB 00:00 2024-03-23T23:29:26.093 INFO:teuthology.orchestra.run.smithi174.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-03-23T23:29:26.118 INFO:teuthology.orchestra.run.smithi174.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-03-23T23:29:26.144 INFO:teuthology.orchestra.run.smithi174.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-23T23:29:26.194 INFO:teuthology.orchestra.run.smithi174.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-03-23T23:29:26.219 INFO:teuthology.orchestra.run.smithi174.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-23T23:29:26.244 INFO:teuthology.orchestra.run.smithi174.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-03-23T23:29:26.269 INFO:teuthology.orchestra.run.smithi174.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-23T23:29:26.337 INFO:teuthology.orchestra.run.smithi174.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-23T23:29:26.379 INFO:teuthology.orchestra.run.smithi174.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-23T23:29:26.570 INFO:teuthology.orchestra.run.smithi174.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-23T23:29:26.646 INFO:teuthology.orchestra.run.smithi174.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 25 MB/s | 1.8 MB 00:00 2024-03-23T23:29:26.688 INFO:teuthology.orchestra.run.smithi174.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-23T23:29:26.729 INFO:teuthology.orchestra.run.smithi174.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-23T23:29:26.780 INFO:teuthology.orchestra.run.smithi174.stdout:(156/174): librados2-19.0.0-2145.g3be9a5c9.el9. 6.4 MB/s | 3.4 MB 00:00 2024-03-23T23:29:26.805 INFO:teuthology.orchestra.run.smithi174.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 949 kB/s | 71 kB 00:00 2024-03-23T23:29:26.839 INFO:teuthology.orchestra.run.smithi174.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 624 kB/s | 36 kB 00:00 2024-03-23T23:29:26.864 INFO:teuthology.orchestra.run.smithi174.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-23T23:29:26.897 INFO:teuthology.orchestra.run.smithi174.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 697 kB/s | 23 kB 00:00 2024-03-23T23:29:26.923 INFO:teuthology.orchestra.run.smithi174.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-03-23T23:29:26.948 INFO:teuthology.orchestra.run.smithi174.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-23T23:29:26.965 INFO:teuthology.orchestra.run.smithi174.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-03-23T23:29:26.990 INFO:teuthology.orchestra.run.smithi174.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-23T23:29:27.015 INFO:teuthology.orchestra.run.smithi174.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-23T23:29:27.032 INFO:teuthology.orchestra.run.smithi174.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 685 kB/s | 29 kB 00:00 2024-03-23T23:29:27.108 INFO:teuthology.orchestra.run.smithi174.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-23T23:29:27.150 INFO:teuthology.orchestra.run.smithi174.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-23T23:29:27.200 INFO:teuthology.orchestra.run.smithi174.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-03-23T23:29:27.250 INFO:teuthology.orchestra.run.smithi174.stdout:(170/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-03-23T23:29:27.275 INFO:teuthology.orchestra.run.smithi174.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 415 kB/s | 31 kB 00:00 2024-03-23T23:29:27.306 INFO:teuthology.orchestra.run.smithi099.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 713 kB/s | 1.0 MB 00:01 2024-03-23T23:29:27.317 INFO:teuthology.orchestra.run.smithi174.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-23T23:29:27.409 INFO:teuthology.orchestra.run.smithi174.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-23T23:29:27.415 INFO:teuthology.orchestra.run.smithi099.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 431 kB/s | 46 kB 00:00 2024-03-23T23:29:27.524 INFO:teuthology.orchestra.run.smithi099.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 732 kB/s | 79 kB 00:00 2024-03-23T23:29:27.551 INFO:teuthology.orchestra.run.smithi174.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 270 kB/s | 74 kB 00:00 2024-03-23T23:29:27.556 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-23T23:29:27.556 INFO:teuthology.orchestra.run.smithi174.stdout:Total 12 MB/s | 224 MB 00:17 2024-03-23T23:29:27.599 INFO:teuthology.orchestra.run.smithi099.stdout:(133/174): python3-grpcio-1.46.7-10.el9.x86_64. 760 kB/s | 2.0 MB 00:02 2024-03-23T23:29:27.633 INFO:teuthology.orchestra.run.smithi099.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-03-23T23:29:27.741 INFO:teuthology.orchestra.run.smithi099.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-23T23:29:27.928 INFO:teuthology.orchestra.run.smithi099.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 483 kB/s | 90 kB 00:00 2024-03-23T23:29:28.020 INFO:teuthology.orchestra.run.smithi099.stdout:(137/174): libarrow-9.0.0-10.el9.x86_64.rpm 785 kB/s | 4.4 MB 00:05 2024-03-23T23:29:28.042 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-23T23:29:28.046 INFO:teuthology.orchestra.run.smithi099.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 612 kB/s | 272 kB 00:00 2024-03-23T23:29:28.071 INFO:teuthology.orchestra.run.smithi099.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-03-23T23:29:28.130 INFO:teuthology.orchestra.run.smithi099.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 281 kB/s | 31 kB 00:00 2024-03-23T23:29:28.180 INFO:teuthology.orchestra.run.smithi099.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-03-23T23:29:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-03-23T23:29:28.297 INFO:teuthology.orchestra.run.smithi099.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 438 kB/s | 51 kB 00:00 2024-03-23T23:29:28.314 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-23T23:29:28.314 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-23T23:29:28.322 INFO:teuthology.orchestra.run.smithi099.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 681 kB/s | 188 kB 00:00 2024-03-23T23:29:28.481 INFO:teuthology.orchestra.run.smithi099.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 490 kB/s | 90 kB 00:00 2024-03-23T23:29:28.590 INFO:teuthology.orchestra.run.smithi099.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-03-23T23:29:28.615 INFO:teuthology.orchestra.run.smithi099.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 611 kB/s | 230 kB 00:00 2024-03-23T23:29:28.866 INFO:teuthology.orchestra.run.smithi099.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 699 kB/s | 192 kB 00:00 2024-03-23T23:29:28.941 INFO:teuthology.orchestra.run.smithi099.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 692 kB/s | 427 kB 00:00 2024-03-23T23:29:29.541 INFO:teuthology.orchestra.run.smithi099.stdout:(150/174): librados2-19.0.0-2145.g3be9a5c9.el9. 5.0 MB/s | 3.4 MB 00:00 2024-03-23T23:29:29.608 INFO:teuthology.orchestra.run.smithi099.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2024-03-23T23:29:29.750 INFO:teuthology.orchestra.run.smithi099.stdout:(152/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-03-23T23:29:29.992 INFO:teuthology.orchestra.run.smithi099.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 6.0 MB/s | 2.3 MB 00:00 2024-03-23T23:29:30.017 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-23T23:29:30.017 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-23T23:29:30.102 INFO:teuthology.orchestra.run.smithi099.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 5.3 MB/s | 1.8 MB 00:00 2024-03-23T23:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 211 kB/s | 42 kB 00:00 2024-03-23T23:29:30.393 INFO:teuthology.orchestra.run.smithi099.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 857 kB/s | 248 kB 00:00 2024-03-23T23:29:30.418 INFO:teuthology.orchestra.run.smithi099.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 320 kB/s | 71 kB 00:00 2024-03-23T23:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 183 kB/s | 36 kB 00:00 2024-03-23T23:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 975 kB/s | 192 kB 00:00 2024-03-23T23:29:30.794 INFO:teuthology.orchestra.run.smithi099.stdout:(160/174): thrift-0.15.0-2.el9.x86_64.rpm 744 kB/s | 1.6 MB 00:02 2024-03-23T23:29:30.877 INFO:teuthology.orchestra.run.smithi099.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 121 kB/s | 23 kB 00:00 2024-03-23T23:29:30.919 INFO:teuthology.orchestra.run.smithi099.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 787 kB/s | 256 kB 00:00 2024-03-23T23:29:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-23T23:29:31.061 INFO:teuthology.orchestra.run.smithi099.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 409 kB/s | 108 kB 00:00 2024-03-23T23:29:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 566 kB/s | 155 kB 00:00 2024-03-23T23:29:31.214 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T23:29:31.237 INFO:teuthology.orchestra.run.smithi099.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 516 kB/s | 136 kB 00:00 2024-03-23T23:29:31.262 INFO:teuthology.orchestra.run.smithi099.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 322 kB/s | 64 kB 00:00 2024-03-23T23:29:31.291 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T23:29:31.321 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:31.334 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:31.354 INFO:teuthology.orchestra.run.smithi099.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 144 kB/s | 29 kB 00:00 2024-03-23T23:29:31.373 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T23:29:31.427 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T23:29:31.467 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T23:29:31.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T23:29:31.513 INFO:teuthology.orchestra.run.smithi099.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.4 MB/s | 1.2 MB 00:00 2024-03-23T23:29:31.536 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T23:29:31.688 INFO:teuthology.orchestra.run.smithi099.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.0 MB/s | 2.3 MB 00:00 2024-03-23T23:29:31.730 INFO:teuthology.orchestra.run.smithi099.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.1 MB/s | 465 kB 00:00 2024-03-23T23:29:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 2.2 MB 00:00 2024-03-23T23:29:31.866 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T23:29:31.922 INFO:teuthology.orchestra.run.smithi099.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 133 kB/s | 31 kB 00:00 2024-03-23T23:29:31.928 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:31.981 INFO:teuthology.orchestra.run.smithi099.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 836 kB/s | 74 kB 00:00 2024-03-23T23:29:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:32.071 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:32.081 INFO:teuthology.orchestra.run.smithi099.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.6 MB/s | 1.1 MB 00:00 2024-03-23T23:29:32.100 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-23T23:29:32.101 INFO:teuthology.orchestra.run.smithi099.stdout:Total 8.5 MB/s | 224 MB 00:26 2024-03-23T23:29:32.145 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:32.175 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:29:32.211 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T23:29:32.385 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T23:29:32.440 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T23:29:32.467 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:32.571 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:32.593 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:32.641 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:32.672 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T23:29:32.695 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:32.711 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:32.886 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:32.918 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:32.958 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T23:29:33.064 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T23:29:33.163 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:33.213 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:33.290 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:33.331 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:33.545 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-03-23T23:29:33.553 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T23:29:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T23:29:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T23:29:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T23:29:33.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T23:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-03-23T23:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-03-23T23:29:33.930 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T23:29:33.984 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T23:29:34.031 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T23:29:34.096 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T23:29:34.130 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T23:29:34.174 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T23:29:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T23:29:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T23:29:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T23:29:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-23T23:29:34.390 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-23T23:29:34.419 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-23T23:29:34.488 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T23:29:34.533 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T23:29:34.606 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T23:29:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T23:29:34.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T23:29:35.305 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T23:29:35.381 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T23:29:35.450 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T23:29:35.502 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:35.511 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:35.541 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T23:29:35.582 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T23:29:35.620 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-23T23:29:35.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T23:29:35.875 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T23:29:36.044 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T23:29:36.102 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T23:29:36.146 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T23:29:36.373 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T23:29:36.449 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T23:29:36.475 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-03-23T23:29:36.476 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-03-23T23:29:36.504 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T23:29:36.568 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T23:29:36.600 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T23:29:36.636 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T23:29:36.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T23:29:36.719 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T23:29:36.754 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T23:29:36.874 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-23T23:29:36.935 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T23:29:37.013 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T23:29:37.048 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T23:29:37.086 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:37.097 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:37.136 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T23:29:37.164 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T23:29:37.190 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T23:29:37.213 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T23:29:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T23:29:37.243 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T23:29:37.257 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T23:29:37.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T23:29:37.299 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T23:29:37.371 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T23:29:37.410 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T23:29:37.450 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T23:29:37.476 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T23:29:37.501 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T23:29:37.573 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T23:29:37.647 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T23:29:37.683 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:37.704 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T23:29:37.801 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T23:29:37.806 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:37.834 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:37.851 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T23:29:37.902 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T23:29:37.926 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:37.954 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:29:37.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T23:29:37.992 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T23:29:38.140 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T23:29:38.163 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T23:29:38.188 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T23:29:38.210 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T23:29:38.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:38.318 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T23:29:38.343 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:38.354 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T23:29:38.364 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:38.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T23:29:38.416 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T23:29:38.428 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:38.453 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T23:29:38.468 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T23:29:38.487 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T23:29:38.499 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:38.515 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:38.526 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T23:29:38.558 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T23:29:38.588 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T23:29:38.664 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T23:29:38.698 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T23:29:38.708 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:38.732 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T23:29:38.738 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:38.774 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T23:29:38.778 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T23:29:38.808 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T23:29:38.876 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T23:29:38.897 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T23:29:38.956 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:38.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T23:29:39.015 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:39.031 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T23:29:39.062 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T23:29:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:39.109 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T23:29:39.141 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:39.208 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T23:29:39.255 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T23:29:39.344 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:39.447 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T23:29:39.541 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T23:29:39.571 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T23:29:39.598 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T23:29:39.633 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T23:29:39.707 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T23:29:39.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T23:29:39.800 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T23:29:39.863 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T23:29:39.898 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T23:29:39.942 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T23:29:39.974 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T23:29:39.988 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T23:29:40.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T23:29:40.036 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T23:29:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T23:29:40.070 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T23:29:40.107 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-23T23:29:40.118 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T23:29:40.150 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-23T23:29:40.179 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-23T23:29:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T23:29:40.249 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T23:29:40.293 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T23:29:40.301 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T23:29:40.368 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T23:29:40.474 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T23:29:40.498 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T23:29:40.502 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T23:29:40.545 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T23:29:40.579 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T23:29:41.058 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T23:29:41.132 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T23:29:41.213 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T23:29:41.219 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T23:29:41.252 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:41.260 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:41.292 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T23:29:41.333 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T23:29:41.341 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T23:29:41.378 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-23T23:29:41.408 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T23:29:41.634 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T23:29:41.791 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T23:29:41.852 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T23:29:41.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T23:29:42.020 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T23:29:42.056 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T23:29:42.111 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T23:29:42.176 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T23:29:42.221 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T23:29:42.258 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T23:29:42.303 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T23:29:42.340 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T23:29:42.383 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T23:29:42.425 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T23:29:42.551 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T23:29:42.557 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T23:29:42.598 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T23:29:42.627 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T23:29:42.645 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T23:29:42.668 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T23:29:42.676 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T23:29:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T23:29:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T23:29:42.832 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T23:29:42.860 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T23:29:42.900 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T23:29:42.926 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T23:29:42.951 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T23:29:43.022 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T23:29:43.046 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T23:29:43.081 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:29:43.168 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T23:29:43.172 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:29:43.203 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-23T23:29:43.239 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T23:29:43.266 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T23:29:43.308 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T23:29:43.356 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T23:29:43.412 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T23:29:43.602 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T23:29:43.645 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T23:29:43.699 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T23:29:43.726 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:29:43.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T23:29:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:29:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-23T23:29:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-03-23T23:29:44.012 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T23:29:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T23:29:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T23:29:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:44.133 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T23:29:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T23:29:44.165 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T23:29:44.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T23:29:44.236 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T23:29:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T23:29:44.275 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T23:29:44.306 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T23:29:44.336 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T23:29:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T23:29:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T23:29:44.414 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T23:29:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T23:29:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T23:29:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T23:29:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T23:29:44.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T23:29:44.572 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T23:29:44.653 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T23:29:44.750 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T23:29:44.794 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T23:29:44.826 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T23:29:44.873 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T23:29:44.927 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T23:29:44.968 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:44.970 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T23:29:45.027 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T23:29:45.117 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:29:45.169 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:29:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:29:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T23:29:45.715 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T23:29:45.743 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T23:29:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T23:29:45.791 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T23:29:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:45.824 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T23:29:45.872 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T23:29:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:29:45.951 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T23:29:45.975 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:45.983 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T23:29:46.014 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:46.050 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:46.072 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-23T23:29:46.127 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T23:29:46.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:46.153 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T23:29:46.181 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T23:29:46.213 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T23:29:46.258 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T23:29:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T23:29:46.588 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:46.620 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:29:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T23:29:46.701 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:29:46.701 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T23:29:46.701 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T23:29:46.701 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:29:46.729 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:29:46.892 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T23:29:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T23:29:47.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T23:29:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T23:29:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:47.853 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T23:29:48.044 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T23:29:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T23:29:48.201 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T23:29:48.215 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T23:29:48.237 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T23:29:48.281 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T23:29:48.293 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T23:29:48.312 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T23:29:48.343 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T23:29:48.370 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T23:29:48.405 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T23:29:48.464 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T23:29:48.508 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T23:29:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T23:29:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T23:29:48.683 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T23:29:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T23:29:48.709 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:29:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T23:29:48.783 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:29:48.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T23:29:48.813 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-23T23:29:48.849 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T23:29:48.893 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T23:29:48.929 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-23T23:29:48.987 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-23T23:29:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-23T23:29:49.135 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T23:29:49.220 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T23:29:49.312 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T23:29:49.324 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T23:29:49.345 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:29:49.399 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:29:49.422 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-23T23:29:49.495 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T23:29:49.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T23:29:50.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T23:29:50.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T23:29:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T23:29:50.489 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:50.514 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T23:29:50.561 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T23:29:50.621 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T23:29:50.672 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-23T23:29:50.713 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T23:29:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T23:29:51.286 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T23:29:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T23:29:51.415 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T23:29:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T23:29:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T23:29:51.674 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:51.709 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:51.737 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T23:29:51.823 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T23:29:51.864 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T23:29:51.902 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T23:29:51.967 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T23:29:52.014 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T23:29:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T23:29:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T23:29:52.247 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:29:52.279 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:29:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T23:29:52.343 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:29:52.343 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T23:29:52.343 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T23:29:52.343 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:29:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T23:29:52.389 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:29:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T23:29:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T23:29:52.589 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T23:29:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T23:29:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T23:29:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T23:29:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T23:29:52.928 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T23:29:53.125 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T23:29:53.258 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T23:29:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T23:29:53.375 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T23:29:53.450 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T23:29:53.696 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T23:29:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T23:29:53.952 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T23:29:54.002 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T23:29:54.045 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T23:29:54.091 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T23:29:54.141 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T23:29:54.196 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T23:29:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T23:29:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T23:29:54.382 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T23:29:54.515 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T23:29:54.557 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T23:29:54.591 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T23:29:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T23:29:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T23:29:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T23:29:54.993 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T23:29:55.049 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T23:29:55.086 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T23:29:55.143 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T23:29:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T23:29:55.351 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T23:29:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T23:29:56.333 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T23:29:56.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T23:29:56.422 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T23:29:56.542 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T23:29:56.593 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T23:29:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T23:29:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T23:29:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T23:29:58.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T23:29:58.275 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-03-23T23:29:58.357 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-03-23T23:29:58.358 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-03-23T23:29:58.358 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-03-23T23:29:58.358 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:29:58.527 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:29:58.542 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:29:58.542 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:29:58.542 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T23:29:58.543 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:29:59.784 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T23:29:59.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T23:29:59.914 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T23:29:59.947 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T23:29:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T23:30:00.006 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:00.035 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:00.035 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:00.036 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T23:30:00.036 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:00.036 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:00.036 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:00.256 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:00.280 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:00.372 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T23:30:00.419 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:30:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-23T23:30:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-23T23:30:00.629 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T23:30:00.723 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:00.746 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:00.747 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:00.747 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T23:30:00.747 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:00.747 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:00.747 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:00.842 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T23:30:01.148 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T23:30:01.176 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:30:01.207 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:01.231 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-23T23:30:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-23T23:30:01.291 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-23T23:30:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-23T23:30:01.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:01.558 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:01.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:02.699 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:02.853 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-23T23:30:02.884 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:02.907 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:02.907 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T23:30:02.907 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:02.908 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:03.000 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-23T23:30:03.772 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:30:03.772 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-03-23T23:30:03.773 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:03.876 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T23:30:03.907 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:30:03.912 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:03.934 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:03.935 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:03.950 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:30:03.950 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:03.950 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T23:30:03.950 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:30:03.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:04.001 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:04.001 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:04.001 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T23:30:04.001 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:04.047 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:04.069 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:04.381 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:04.404 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:04.405 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:04.405 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T23:30:04.405 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:04.405 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:04.405 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-23T23:30:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:30:04.778 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-23T23:30:04.778 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T23:30:04.778 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T23:30:04.778 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:04.831 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:30:05.437 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:05.464 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:05.688 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:05.711 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:06.155 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:06.178 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:06.178 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:06.178 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T23:30:06.178 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:06.178 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:06.179 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:06.265 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T23:30:06.649 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:06.680 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:06.712 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-23T23:30:06.771 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-23T23:30:06.803 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:07.038 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:07.071 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:08.032 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:08.054 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:08.155 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:08.317 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-23T23:30:08.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:08.362 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:08.363 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:08.363 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T23:30:08.363 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:08.363 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:08.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:08.398 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:08.410 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:08.455 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-23T23:30:08.835 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:08.876 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-23T23:30:08.907 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-23T23:30:08.934 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T23:30:09.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-23T23:30:09.228 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T23:30:09.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-23T23:30:09.315 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T23:30:09.328 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T23:30:09.359 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:09.376 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T23:30:09.381 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:09.382 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:09.382 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T23:30:09.382 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:09.382 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:09.382 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:09.419 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T23:30:09.426 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:09.448 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:09.448 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:09.448 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T23:30:09.448 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:09.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-23T23:30:09.476 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T23:30:09.493 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:09.505 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:09.516 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:09.551 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T23:30:09.612 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T23:30:09.645 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T23:30:09.819 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:09.842 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:09.842 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:09.842 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T23:30:09.842 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:09.842 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:09.843 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:10.201 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:10.236 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:10.255 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:11.094 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-23T23:30:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T23:30:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T23:30:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T23:30:11.343 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T23:30:11.368 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T23:30:11.393 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T23:30:11.420 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T23:30:11.420 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:11.499 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:11.529 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T23:30:11.529 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:11.614 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:11.640 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T23:30:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T23:30:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T23:30:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T23:30:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T23:30:11.796 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T23:30:11.829 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T23:30:11.854 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T23:30:11.879 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T23:30:11.904 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:11.930 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T23:30:11.936 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:11.937 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:12.124 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:12.124 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:12.216 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:12.240 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:12.272 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T23:30:12.300 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T23:30:12.325 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T23:30:12.389 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:14.290 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:14.331 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-23T23:30:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-23T23:30:14.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T23:30:14.567 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-23T23:30:14.680 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T23:30:14.720 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-23T23:30:14.773 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T23:30:14.828 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T23:30:14.862 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T23:30:14.887 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-23T23:30:14.919 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T23:30:14.949 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T23:30:14.995 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T23:30:15.047 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T23:30:15.072 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T23:30:15.642 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:15.678 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:15.698 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:16.518 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-23T23:30:16.598 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T23:30:16.630 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T23:30:16.719 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T23:30:16.759 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T23:30:16.784 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T23:30:16.809 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T23:30:16.836 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T23:30:16.836 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:16.924 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:16.953 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T23:30:16.953 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:17.038 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:17.064 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T23:30:17.097 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T23:30:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T23:30:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T23:30:17.181 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T23:30:17.228 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T23:30:17.269 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T23:30:17.294 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T23:30:17.318 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:17.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-23T23:30:17.318 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-23T23:30:17.319 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-23T23:30:17.320 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T23:30:17.321 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T23:30:17.322 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T23:30:17.323 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T23:30:17.324 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T23:30:17.325 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T23:30:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T23:30:17.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T23:30:17.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T23:30:17.329 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T23:30:17.330 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T23:30:17.331 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T23:30:17.332 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T23:30:17.333 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T23:30:17.344 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:17.369 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T23:30:17.376 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:17.377 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:17.555 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:17.555 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:17.638 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:17.663 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:17.696 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T23:30:17.723 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T23:30:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-03-23T23:30:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:17.748 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T23:30:17.812 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:17.974 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:30:18.017 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-23T23:30:18.017 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:18.017 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T23:30:18.017 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T23:30:19.735 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:19.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-23T23:30:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T23:30:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T23:30:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.339 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.340 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.341 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T23:30:20.342 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-16.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.343 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T23:30:20.344 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T23:30:20.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T23:30:20.346 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T23:30:20.347 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T23:30:20.348 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T23:30:20.349 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T23:30:20.350 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T23:30:20.351 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T23:30:20.352 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-23T23:30:20.352 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-23T23:30:20.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T23:30:20.606 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T23:30:20.843 DEBUG:teuthology.parallel:result is None 2024-03-23T23:30:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-23T23:30:21.447 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-23T23:30:21.548 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-23T23:30:21.576 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:21.907 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-23T23:30:21.952 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:22.675 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:22.675 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-23T23:30:22.675 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-23T23:30:22.676 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-23T23:30:22.677 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:30:22.678 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-23T23:30:22.678 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-23T23:30:22.678 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-23T23:30:22.678 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-23T23:30:22.678 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-23T23:30:22.679 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-23T23:30:22.680 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-23T23:30:22.681 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T23:30:22.682 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T23:30:22.683 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T23:30:22.684 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T23:30:22.685 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T23:30:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T23:30:22.687 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T23:30:22.688 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T23:30:22.689 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T23:30:22.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T23:30:22.692 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T23:30:22.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:22.694 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T23:30:22.694 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T23:30:22.694 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T23:30:23.073 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-23T23:30:23.111 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-23T23:30:23.424 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-23T23:30:23.455 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-23T23:30:23.523 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:23.523 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T23:30:23.523 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:23.523 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T23:30:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-23T23:30:23.700 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-23T23:30:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T23:30:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T23:30:24.810 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:24.935 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-23T23:30:24.935 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:24.935 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T23:30:24.935 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:25.083 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:25.083 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:25.083 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T23:30:25.083 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:25.084 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T23:30:25.084 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:25.476 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T23:30:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:25.764 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T23:30:25.764 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.765 INFO:teuthology.orchestra.run.smithi174.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-23T23:30:25.766 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.767 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:25.768 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: jq-1.6-16.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.769 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T23:30:25.770 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T23:30:25.771 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T23:30:25.772 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T23:30:25.773 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T23:30:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T23:30:25.775 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T23:30:25.776 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.777 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:30:25.778 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-23T23:30:26.292 DEBUG:teuthology.parallel:result is None 2024-03-23T23:30:30.374 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:30.428 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-23T23:30:30.488 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-23T23:30:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T23:30:30.829 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-23T23:30:30.958 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T23:30:31.013 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-23T23:30:31.090 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T23:30:31.156 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T23:30:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T23:30:31.256 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-23T23:30:31.306 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T23:30:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T23:30:31.440 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T23:30:31.549 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T23:30:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T23:30:32.477 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:32.521 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:32.576 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T23:30:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-23T23:30:34.009 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T23:30:34.072 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T23:30:34.240 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T23:30:34.295 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T23:30:34.337 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T23:30:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T23:30:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T23:30:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:34.565 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T23:30:34.608 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T23:30:34.608 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:34.796 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T23:30:34.855 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T23:30:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T23:30:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T23:30:34.968 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T23:30:35.043 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T23:30:35.116 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T23:30:35.173 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T23:30:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T23:30:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T23:30:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:35.340 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T23:30:35.376 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:35.376 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T23:30:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T23:30:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:35.955 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T23:30:35.987 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T23:30:36.027 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T23:30:36.148 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T23:30:36.391 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-23T23:30:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-23T23:30:41.216 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-23T23:30:41.217 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-23T23:30:41.218 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T23:30:41.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T23:30:41.220 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T23:30:41.220 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T23:30:41.220 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T23:30:41.220 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T23:30:41.220 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T23:30:41.221 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T23:30:41.222 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T23:30:41.223 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T23:30:41.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T23:30:41.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T23:30:41.226 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T23:30:41.227 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T23:30:41.228 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T23:30:41.229 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T23:30:41.230 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T23:30:41.231 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T23:30:41.232 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T23:30:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T23:30:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-16.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T23:30:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T23:30:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T23:30:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T23:30:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T23:30:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T23:30:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T23:30:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T23:30:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T23:30:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T23:30:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T23:30:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T23:30:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T23:30:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:30:43.713 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-23T23:30:44.378 DEBUG:teuthology.parallel:result is None 2024-03-23T23:30:44.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:44.531 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T23:30:44.583 INFO:teuthology.orchestra.run.smithi099.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.583 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.583 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-23T23:30:44.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:44.726 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T23:30:44.751 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.751 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.751 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-23T23:30:44.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:44.892 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T23:30:44.918 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.919 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-23T23:30:44.919 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-23T23:30:44.921 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T23:30:44.921 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:30:44.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T23:30:45.010 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:30:45.010 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T23:30:45.043 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:30:45.044 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T23:30:45.078 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T23:30:45.078 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:30:45.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T23:30:45.154 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T23:30:45.268 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:30:45.268 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T23:30:45.300 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T23:30:45.370 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:30:45.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T23:30:45.401 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T23:30:45.470 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T23:30:45.470 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:30:45.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T23:30:45.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T23:30:45.665 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:30:45.665 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T23:30:45.692 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T23:30:45.759 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:30:45.759 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T23:30:45.786 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T23:30:45.853 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T23:30:45.854 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:30:45.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T23:30:45.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T23:30:46.037 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:30:46.037 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T23:30:46.065 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T23:30:46.132 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:30:46.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T23:30:46.159 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T23:30:46.226 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-23T23:30:46.324 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-23T23:30:46.325 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:46.325 INFO:tasks.cephadm:Cluster fsid is 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:30:46.325 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-23T23:30:46.325 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.99', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.174'} 2024-03-23T23:30:46.325 INFO:tasks.cephadm:First mon is mon.a on smithi099 2024-03-23T23:30:46.325 INFO:tasks.cephadm:First mgr is x 2024-03-23T23:30:46.326 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-23T23:30:46.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-03-23T23:30:46.401 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-03-23T23:30:46.428 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-03-23T23:30:46.454 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-23T23:30:46.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:46.673 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-23T23:30:46.886 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:30:46.887 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-23T23:30:46.888 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:47.386 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 23 23:30 /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:47.386 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:47.980 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 23 23:30 /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:47.980 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:48.568 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 23 23:30 /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:48.568 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:48.617 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:48.635 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T23:30:48.669 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-23T23:30:48.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-23T23:30:48.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-23T23:30:48.682 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-23T23:30:49.504 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-23T23:30:49.519 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-23T23:30:50.274 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-23T23:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-23T23:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-23T23:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-03-23T23:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-23T23:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:31:33.461 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-23T23:31:33.462 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-23T23:31:33.462 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-23T23:31:33.462 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-03-23T23:31:33.463 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-23T23:31:33.463 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-03-23T23:31:33.463 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-03-23T23:31:36.683 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-23T23:31:36.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-03-23T23:31:36.804 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-03-23T23:31:36.839 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-03-23T23:31:36.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-03-23T23:31:36.954 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-03-23T23:31:36.985 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-03-23T23:31:37.013 INFO:tasks.cephadm:Writing seed config... 2024-03-23T23:31:37.014 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-23T23:31:37.014 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-23T23:31:37.014 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-23T23:31:37.015 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-23T23:31:37.016 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-23T23:31:37.017 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-23T23:31:37.018 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:31:37.018 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-23T23:31:37.072 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5fd03d00-e96d-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-23T23:31:37.072 DEBUG:teuthology.orchestra.run.smithi099:mon.a> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service 2024-03-23T23:31:37.077 DEBUG:teuthology.orchestra.run.smithi099:mgr.x> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x.service 2024-03-23T23:31:37.121 INFO:tasks.cephadm:Bootstrapping... 2024-03-23T23:31:37.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.99 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:31:37.391 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-23T23:31:37.392 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '5fd03d00-e96d-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.99', '--skip-admin-label'] 2024-03-23T23:31:37.447 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T23:31:37.448 INFO:teuthology.orchestra.run.smithi099.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-23T23:31:37.448 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying podman|docker is present... 2024-03-23T23:31:37.502 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T23:31:37.502 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying lvm2 is present... 2024-03-23T23:31:37.503 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying time synchronization is in place... 2024-03-23T23:31:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T23:31:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T23:31:37.537 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T23:31:37.537 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:31:37.559 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-03-23T23:31:37.577 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-03-23T23:31:37.577 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-03-23T23:31:37.578 INFO:teuthology.orchestra.run.smithi099.stdout:Repeating the final host check... 2024-03-23T23:31:37.631 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T23:31:37.631 INFO:teuthology.orchestra.run.smithi099.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-23T23:31:37.631 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl is present 2024-03-23T23:31:37.631 INFO:teuthology.orchestra.run.smithi099.stdout:lvcreate is present 2024-03-23T23:31:37.650 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T23:31:37.650 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T23:31:37.670 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T23:31:37.670 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:31:37.692 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-03-23T23:31:37.712 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-03-23T23:31:37.712 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-03-23T23:31:37.712 INFO:teuthology.orchestra.run.smithi099.stdout:Host looks OK 2024-03-23T23:31:37.713 INFO:teuthology.orchestra.run.smithi099.stdout:Cluster fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:31:37.713 INFO:teuthology.orchestra.run.smithi099.stdout:Acquiring lock 139961541627328 on /run/cephadm/5fd03d00-e96d-11ee-95cd-87774f69a715.lock 2024-03-23T23:31:37.713 INFO:teuthology.orchestra.run.smithi099.stdout:Lock 139961541627328 acquired on /run/cephadm/5fd03d00-e96d-11ee-95cd-87774f69a715.lock 2024-03-23T23:31:37.715 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 3300 ... 2024-03-23T23:31:37.716 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 6789 ... 2024-03-23T23:31:37.716 INFO:teuthology.orchestra.run.smithi099.stdout:Base mon IP(s) is [172.21.15.99:3300, 172.21.15.99:6789], mon addrv is [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-03-23T23:31:37.730 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.99 metric 100 2024-03-23T23:31:37.730 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.99 metric 100 2024-03-23T23:31:37.743 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-23T23:31:37.743 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-23T23:31:37.743 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-03-23T23:31:37.753 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-23T23:31:37.754 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-23T23:31:37.754 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T23:31:37.754 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-23T23:31:37.754 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-23T23:31:37.754 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T23:31:37.757 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-03-23T23:31:37.757 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-03-23T23:31:37.757 INFO:teuthology.orchestra.run.smithi099.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-23T23:31:37.760 INFO:teuthology.orchestra.run.smithi099.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-23T23:31:37.762 INFO:teuthology.orchestra.run.smithi099.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-23T23:31:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-23T23:31:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-23T23:31:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Getting image source signatures 2024-03-23T23:31:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-23T23:31:42.699 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-23T23:31:42.700 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-23T23:31:42.700 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-23T23:31:43.986 INFO:teuthology.orchestra.run.smithi099.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-23T23:31:43.986 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-23T23:31:43.986 INFO:teuthology.orchestra.run.smithi099.stdout:Extracting ceph user uid/gid from container image... 2024-03-23T23:31:45.178 INFO:teuthology.orchestra.run.smithi099.stdout:stat: stdout 167 167 2024-03-23T23:31:45.178 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial keys... 2024-03-23T23:31:46.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQBhZv9lAtq/JhAAK9Kn+L7Iimcgl/4KM9vy1A== 2024-03-23T23:31:47.384 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQBiZv9l72yCKxAAkTw7yeL+vIPdHI4rXubahg== 2024-03-23T23:31:50.085 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQBlZv9lB4LAFhAApXNNH1fYcBnOl0IjazJPbQ== 2024-03-23T23:31:50.087 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial monmap... 2024-03-23T23:31:51.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T23:31:51.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-23T23:31:51.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:31:51.217 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool for a [v2:172.21.15.99:3300,v1:172.21.15.99:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = quincy 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: set fsid to 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:31:51.218 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mon... 2024-03-23T23:31:52.393 INFO:teuthology.orchestra.run.smithi099.stdout:create mon.a on 2024-03-23T23:31:52.749 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-23T23:31:52.983 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-23T23:31:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target. 2024-03-23T23:31:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target. 2024-03-23T23:31:53.635 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a 2024-03-23T23:31:53.635 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-23T23:31:53.903 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target.wants/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@.service. 2024-03-23T23:31:55.421 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T23:31:55.421 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-23T23:31:55.435 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T23:31:55.435 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:31:55.435 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-23T23:31:55.435 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T23:31:55.436 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon to start... 2024-03-23T23:31:55.436 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon... 2024-03-23T23:31:56.537 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:56 smithi099 ceph-mon[31340]: from='client.? 172.21.15.99:0/4092659714' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout cluster: 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout id: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout services: 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.801914s) 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout data: 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-23T23:31:56.916 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pgs: 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.093+0000 7faf33800700 1 Processor -- start 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.093+0000 7faf33800700 1 -- start start 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.093+0000 7faf33800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.093+0000 7faf33800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c07be50 con 0x7faf2c106340 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.094+0000 7faf30e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.095+0000 7faf30e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57898/0 (socket says 172.21.15.99:57898) 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.095+0000 7faf30e00700 1 -- 172.21.15.99:0/3120108334 learned_addr learned my addr 172.21.15.99:0/3120108334 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.095+0000 7faf30e00700 1 -- 172.21.15.99:0/3120108334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c106ca0 con 0x7faf2c106340 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.095+0000 7faf30e00700 1 --2- 172.21.15.99:0/3120108334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf200098a0 tx=0x7faf20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fd10ac46af6d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf2b400700 1 -- 172.21.15.99:0/3120108334 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf20004160 con 0x7faf2c106340 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf2b400700 1 -- 172.21.15.99:0/3120108334 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20004300 con 0x7faf2c106340 2024-03-23T23:31:56.917 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf2b400700 1 -- 172.21.15.99:0/3120108334 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf200036a0 con 0x7faf2c106340 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf33800700 1 -- 172.21.15.99:0/3120108334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 msgr2=0x7faf2c106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf33800700 1 --2- 172.21.15.99:0/3120108334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf200098a0 tx=0x7faf20009bb0 comp rx=0 tx=0).stop 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf33800700 1 -- 172.21.15.99:0/3120108334 shutdown_connections 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf33800700 1 --2- 172.21.15.99:0/3120108334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c106760 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.096+0000 7faf33800700 1 -- 172.21.15.99:0/3120108334 >> 172.21.15.99:0/3120108334 conn(0x7faf2c101f20 msgr2=0x7faf2c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.097+0000 7faf33800700 1 -- 172.21.15.99:0/3120108334 shutdown_connections 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.097+0000 7faf33800700 1 -- 172.21.15.99:0/3120108334 wait complete. 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf33800700 1 Processor -- start 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf33800700 1 -- start start 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf33800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf33800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1071d0 con 0x7faf2c106340 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf30e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:56.918 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf30e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57900/0 (socket says 172.21.15.99:57900) 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.098+0000 7faf30e00700 1 -- 172.21.15.99:0/4092659714 learned_addr learned my addr 172.21.15.99:0/4092659714 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf30e00700 1 -- 172.21.15.99:0/4092659714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c19d8c0 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf30e00700 1 --2- 172.21.15.99:0/4092659714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf20000c00 tx=0x7faf20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf20003db0 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20011630 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c19db50 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.099+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf20011950 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.100+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c1a0840 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.100+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faf2001a460 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.100+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faf2001b780 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.101+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c069250 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.104+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faf2001ba20 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.186+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7faf2c1a0a10 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.187+0000 7faf29600700 1 -- 172.21.15.99:0/4092659714 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7faf2001bc00 con 0x7faf2c106340 2024-03-23T23:31:56.919 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 msgr2=0x7faf2c19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 --2- 172.21.15.99:0/4092659714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf20000c00 tx=0x7faf20004750 comp rx=0 tx=0).stop 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 shutdown_connections 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 --2- 172.21.15.99:0/4092659714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf2c106340 0x7faf2c19d380 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 >> 172.21.15.99:0/4092659714 conn(0x7faf2c101f20 msgr2=0x7faf2c1939c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 shutdown_connections 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:56.188+0000 7faf33800700 1 -- 172.21.15.99:0/4092659714 wait complete. 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:mon is available 2024-03-23T23:31:56.920 INFO:teuthology.orchestra.run.smithi099.stdout:Assimilating anything we can from ceph.conf... 2024-03-23T23:31:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:57 smithi099 ceph-mon[31340]: from='client.? 172.21.15.99:0/1193604740' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T23:31:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:57 smithi099 ceph-mon[31340]: from='client.? 172.21.15.99:0/1193604740' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-23T23:31:58.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T23:31:58.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.482+0000 7f9413e00700 1 Processor -- start 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.483+0000 7f9413e00700 1 -- start start 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.483+0000 7f9413e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.483+0000 7f9413e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0824e0 con 0x7f940c081af0 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.483+0000 7f9411400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.484+0000 7f9411400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57912/0 (socket says 172.21.15.99:57912) 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.484+0000 7f9411400700 1 -- 172.21.15.99:0/3031826183 learned_addr learned my addr 172.21.15.99:0/3031826183 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.484+0000 7f9411400700 1 -- 172.21.15.99:0/3031826183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c10f270 con 0x7f940c081af0 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.484+0000 7f9411400700 1 --2- 172.21.15.99:0/3031826183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94000098a0 tx=0x7f9400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c450fc83b0a3a25f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:58.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f940be00700 1 -- 172.21.15.99:0/3031826183 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400004160 con 0x7f940c081af0 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f940be00700 1 -- 172.21.15.99:0/3031826183 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400004300 con 0x7f940c081af0 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f940be00700 1 -- 172.21.15.99:0/3031826183 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94000036a0 con 0x7f940c081af0 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 -- 172.21.15.99:0/3031826183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 msgr2=0x7f940c081f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 --2- 172.21.15.99:0/3031826183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94000098a0 tx=0x7f9400009bb0 comp rx=0 tx=0).stop 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 -- 172.21.15.99:0/3031826183 shutdown_connections 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 --2- 172.21.15.99:0/3031826183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c081f10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 -- 172.21.15.99:0/3031826183 >> 172.21.15.99:0/3031826183 conn(0x7f940c07d140 msgr2=0x7f940c07f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.485+0000 7f9413e00700 1 -- 172.21.15.99:0/3031826183 shutdown_connections 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.486+0000 7f9413e00700 1 -- 172.21.15.99:0/3031826183 wait complete. 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.486+0000 7f9413e00700 1 Processor -- start 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9413e00700 1 -- start start 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9413e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9413e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c10f7a0 con 0x7f940c081af0 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9411400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:58.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9411400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57914/0 (socket says 172.21.15.99:57914) 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9411400700 1 -- 172.21.15.99:0/1193604740 learned_addr learned my addr 172.21.15.99:0/1193604740 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.487+0000 7f9411400700 1 -- 172.21.15.99:0/1193604740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c1a1cb0 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f9411400700 1 --2- 172.21.15.99:0/1193604740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9400000c00 tx=0x7f9400004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400003db0 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400011630 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400011950 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c1a1f40 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c1a4c30 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.488+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9400011af0 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.489+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f940001c450 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.489+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c069250 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.492+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9400032020 con 0x7f940c081af0 2024-03-23T23:31:58.249 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.568+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f940c02dc40 con 0x7f940c081af0 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.576+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f9400004260 con 0x7f940c081af0 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f940a000700 1 -- 172.21.15.99:0/1193604740 <== mon.0 v2:172.21.15.99:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f940001cb60 con 0x7f940c081af0 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 msgr2=0x7f940c1a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f9413e00700 1 --2- 172.21.15.99:0/1193604740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9400000c00 tx=0x7f9400004750 comp rx=0 tx=0).stop 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 shutdown_connections 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f9413e00700 1 --2- 172.21.15.99:0/1193604740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f940c081af0 0x7f940c1a1770 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.578+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 >> 172.21.15.99:0/1193604740 conn(0x7f940c07d140 msgr2=0x7f940c198470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.579+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 shutdown_connections 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:57.579+0000 7f9413e00700 1 -- 172.21.15.99:0/1193604740 wait complete. 2024-03-23T23:31:58.250 INFO:teuthology.orchestra.run.smithi099.stdout:Generating new minimal ceph.conf... 2024-03-23T23:31:59.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:58 smithi099 ceph-mon[31340]: from='client.? 172.21.15.99:0/2922195216' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.786+0000 7f862e000700 1 Processor -- start 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.786+0000 7f862e000700 1 -- start start 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.787+0000 7f862e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.787+0000 7f862e000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862807beb0 con 0x7f86281062e0 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.787+0000 7f8627400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:59.559 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.788+0000 7f8627400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57922/0 (socket says 172.21.15.99:57922) 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.788+0000 7f8627400700 1 -- 172.21.15.99:0/1078441428 learned_addr learned my addr 172.21.15.99:0/1078441428 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.788+0000 7f8627400700 1 -- 172.21.15.99:0/1078441428 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628106c40 con 0x7f86281062e0 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.788+0000 7f8627400700 1 --2- 172.21.15.99:0/1078441428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86180098a0 tx=0x7f8618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f5fe4dd8403776 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f8626000700 1 -- 172.21.15.99:0/1078441428 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8618004160 con 0x7f86281062e0 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f8626000700 1 -- 172.21.15.99:0/1078441428 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8618004300 con 0x7f86281062e0 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f8626000700 1 -- 172.21.15.99:0/1078441428 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8618011b50 con 0x7f86281062e0 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 -- 172.21.15.99:0/1078441428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 msgr2=0x7f8628106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 --2- 172.21.15.99:0/1078441428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86180098a0 tx=0x7f8618009bb0 comp rx=0 tx=0).stop 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 -- 172.21.15.99:0/1078441428 shutdown_connections 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 --2- 172.21.15.99:0/1078441428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f8628106700 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 -- 172.21.15.99:0/1078441428 >> 172.21.15.99:0/1078441428 conn(0x7f8628101f20 msgr2=0x7f8628104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 -- 172.21.15.99:0/1078441428 shutdown_connections 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.789+0000 7f862e000700 1 -- 172.21.15.99:0/1078441428 wait complete. 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.790+0000 7f862e000700 1 Processor -- start 2024-03-23T23:31:59.560 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f862e000700 1 -- start start 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f862e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f862e000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628107cd0 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f8627400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f8627400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57928/0 (socket says 172.21.15.99:57928) 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f8627400700 1 -- 172.21.15.99:0/2922195216 learned_addr learned my addr 172.21.15.99:0/2922195216 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.791+0000 7f8627400700 1 -- 172.21.15.99:0/2922195216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862819e1b0 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f8627400700 1 --2- 172.21.15.99:0/2922195216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8618011b60 tx=0x7f8618011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8618003970 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8618003b10 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862819e440 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8618024610 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.792+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86281a1130 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.793+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8618024c50 con 0x7f86281062e0 2024-03-23T23:31:59.561 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.793+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8618011e00 con 0x7f86281062e0 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.794+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8628069250 con 0x7f86281062e0 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.796+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8618043dd0 con 0x7f86281062e0 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.872+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f862802dc80 con 0x7f86281062e0 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.872+0000 7f861fe00700 1 -- 172.21.15.99:0/2922195216 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8618032030 con 0x7f86281062e0 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 msgr2=0x7f862819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 --2- 172.21.15.99:0/2922195216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8618011b60 tx=0x7f8618011c40 comp rx=0 tx=0).stop 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 shutdown_connections 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 --2- 172.21.15.99:0/2922195216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86281062e0 0x7f862819dc70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 >> 172.21.15.99:0/2922195216 conn(0x7f8628101f20 msgr2=0x7f86281948e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 shutdown_connections 2024-03-23T23:31:59.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:31:58.874+0000 7f862e000700 1 -- 172.21.15.99:0/2922195216 wait complete. 2024-03-23T23:31:59.563 INFO:teuthology.orchestra.run.smithi099.stdout:Restarting the monitor... 2024-03-23T23:31:59.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:59 smithi099 systemd[1]: Stopping Ceph mon.a for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:32:00.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a[31335]: 2024-03-23T23:31:59.989+0000 7f63e4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:32:00.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:31:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a[31335]: 2024-03-23T23:31:59.989+0000 7f63e4e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-23T23:32:00.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:00 smithi099 podman[31594]: 2024-03-23 23:32:00.114244616 +0000 UTC m=+0.235734390 container died 83ee7b43785330bbd19825af346ec86d8be1ec35ef7db0559b44e4aaf3e20859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:32:00.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:00 smithi099 podman[31594]: 2024-03-23 23:32:00.566866377 +0000 UTC m=+0.688356145 container cleanup 83ee7b43785330bbd19825af346ec86d8be1ec35ef7db0559b44e4aaf3e20859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-23T23:32:00.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:00 smithi099 bash[31594]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a 2024-03-23T23:32:01.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:00 smithi099 podman[31608]: 2024-03-23 23:32:00.792775316 +0000 UTC m=+0.668627831 container remove 83ee7b43785330bbd19825af346ec86d8be1ec35ef7db0559b44e4aaf3e20859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-23T23:32:01.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-23T23:32:01.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 systemd[1]: Stopped Ceph mon.a for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:32:01.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service: Consumed 1.867s CPU time. 2024-03-23T23:32:01.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 systemd[1]: Starting Ceph mon.a for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:32:01.923 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 podman[31697]: 2024-03-23 23:32:01.574569237 +0000 UTC m=+0.032414067 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:32:01.923 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 podman[31697]: 2024-03-23 23:32:01.693520027 +0000 UTC m=+0.151364879 container create 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-23T23:32:01.923 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 podman[31697]: 2024-03-23 23:32:01.913638567 +0000 UTC m=+0.371483489 container init 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T23:32:01.923 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 podman[31697]: 2024-03-23 23:32:01.923526589 +0000 UTC m=+0.381371409 container start 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-23T23:32:02.063 INFO:teuthology.orchestra.run.smithi099.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-23T23:32:02.218 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T23:32:02.218 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-23T23:32:02.218 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: pidfile_write: ignore empty --pid-file 2024-03-23T23:32:02.218 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: load: jerasure load: lrc 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: RocksDB version: 7.9.2 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Git sha 0 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: DB SUMMARY 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: DB Session ID: S6L2KDU3YN4RLORIS45J 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: CURRENT file: CURRENT 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100158 ; 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.error_if_exists: 0 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.create_if_missing: 0 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.paranoid_checks: 1 2024-03-23T23:32:02.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.env: 0x56091e569b20 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.info_log: 0x560920999960 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.statistics: (nil) 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.use_fsync: 0 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_log_file_size: 0 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T23:32:02.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_fallocate: 1 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.use_direct_reads: 0 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.db_log_dir: 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.wal_dir: 2024-03-23T23:32:02.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T23:32:02.222 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.write_buffer_manager: 0x56092096c1e0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.unordered_write: 0 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T23:32:02.223 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.row_cache: None 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.wal_filter: None 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.two_write_queues: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.wal_compression: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.atomic_flush: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.log_readahead_size: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T23:32:02.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_background_jobs: 2 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_background_compactions: -1 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_subcompactions: 1 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T23:32:02.225 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_open_files: -1 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_background_flushes: -1 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Compression algorithms supported: 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kZSTD supported: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kXpressCompression supported: 0 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kZlibCompression supported: 1 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kSnappyCompression supported: 1 2024-03-23T23:32:02.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kLZ4Compression supported: 1 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: kBZip2Compression supported: 0 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.merge_operator: 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_filter: None 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560920999a60) 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T23:32:02.227 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_type: 0 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_index_type: 0 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_shortening: 1 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: checksum: 4 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: no_block_cache: 0 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache: 0x56092099f090 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_options: 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: capacity : 536870912 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_shard_bits : 4 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: strict_capacity_limit : 0 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_compressed: (nil) 2024-03-23T23:32:02.228 INFO:journalctl@ceph.mon.a.smithi099.stdout: persistent_cache: (nil) 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size: 4096 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size_deviation: 10 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_restart_interval: 16 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_block_restart_interval: 1 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: metadata_block_size: 4096 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: partition_filters: 0 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: use_delta_encoding: 1 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: filter_policy: bloomfilter 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: whole_key_filtering: 1 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: verify_compression: 0 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: format_version: 5 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_index_compression: 1 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_align: 0 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: max_auto_readahead_size: 262144 2024-03-23T23:32:02.229 INFO:journalctl@ceph.mon.a.smithi099.stdout: prepopulate_block_cache: 0 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout: initial_auto_readahead_size: 8192 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression: NoCompression 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.num_levels: 7 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T23:32:02.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T23:32:02.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T23:32:02.232 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T23:32:02.233 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.table_properties_collectors: 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.inplace_update_support: 0 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T23:32:02.234 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.bloom_locality: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.max_successive_merges: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.ttl: 2592000 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enable_blob_files: false 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.min_blob_size: 0 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T23:32:02.235 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 06174a55-499e-486a-a9c5-753639f58da3 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711236721979274, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711236721981288, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95351, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 93337, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13793, "raw_average_key_size": 50, "raw_value_size": 85826, "raw_average_value_size": 312, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711236721, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "06174a55-499e-486a-a9c5-753639f58da3", "db_session_id": "S6L2KDU3YN4RLORIS45J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711236721981452, "job": 1, "event": "recovery_finished"} 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560920a92000 2024-03-23T23:32:02.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: DB pointer 0x560920a7c000 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** DB Stats ** 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 94.95 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T23:32:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 94.95 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 67.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.238 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 13.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 13.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache BinnedLRUCache@0x56092099f090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: starting mon.a rank 0 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???) e1 preinit fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:32:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).mds e1 new map 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).mds e1 print_map 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: e1 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: legacy client fscid: -1 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout: No filesystems configured 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:01 smithi099 ceph-mon[31715]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 bash[31697]: 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 ceph-mon[31715]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 ceph-mon[31715]: monmap e1: 1 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:32:02.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 ceph-mon[31715]: fsmap 2024-03-23T23:32:02.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 ceph-mon[31715]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T23:32:02.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 ceph-mon[31715]: mgrmap e1: no daemons active 2024-03-23T23:32:02.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:02 smithi099 systemd[1]: Started Ceph mon.a for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:32:03.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.819+0000 7f8f2be00700 1 Processor -- start 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.819+0000 7f8f2be00700 1 -- start start 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.819+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.819+0000 7f8f2be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c082270 con 0x7f8f2c081910 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.820+0000 7f8f2aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.820+0000 7f8f2aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49736/0 (socket says 172.21.15.99:49736) 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.820+0000 7f8f2aa00700 1 -- 172.21.15.99:0/1535689295 learned_addr learned my addr 172.21.15.99:0/1535689295 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.820+0000 7f8f2aa00700 1 -- 172.21.15.99:0/1535689295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c10eff0 con 0x7f8f2c081910 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.821+0000 7f8f2aa00700 1 --2- 172.21.15.99:0/1535689295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0098a0 tx=0x7f8f1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605e192a8215b0e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.821+0000 7f8f2a000700 1 -- 172.21.15.99:0/1535689295 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f1c004160 con 0x7f8f2c081910 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.821+0000 7f8f2a000700 1 -- 172.21.15.99:0/1535689295 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f1c004300 con 0x7f8f2c081910 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.821+0000 7f8f2a000700 1 -- 172.21.15.99:0/1535689295 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f1c011b50 con 0x7f8f2c081910 2024-03-23T23:32:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 -- 172.21.15.99:0/1535689295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 msgr2=0x7f8f2c081d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 --2- 172.21.15.99:0/1535689295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0098a0 tx=0x7f8f1c009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 -- 172.21.15.99:0/1535689295 shutdown_connections 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 --2- 172.21.15.99:0/1535689295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c081d30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 -- 172.21.15.99:0/1535689295 >> 172.21.15.99:0/1535689295 conn(0x7f8f2c07cf30 msgr2=0x7f8f2c07f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 -- 172.21.15.99:0/1535689295 shutdown_connections 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.822+0000 7f8f2be00700 1 -- 172.21.15.99:0/1535689295 wait complete. 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.823+0000 7f8f2be00700 1 Processor -- start 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.823+0000 7f8f2be00700 1 -- start start 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.823+0000 7f8f2be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.823+0000 7f8f2be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c112430 con 0x7f8f2c081910 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.824+0000 7f8f2aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.824+0000 7f8f2aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49752/0 (socket says 172.21.15.99:49752) 2024-03-23T23:32:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.824+0000 7f8f2aa00700 1 -- 172.21.15.99:0/2161026751 learned_addr learned my addr 172.21.15.99:0/2161026751 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.824+0000 7f8f2aa00700 1 -- 172.21.15.99:0/2161026751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c1a23e0 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.824+0000 7f8f2aa00700 1 --2- 172.21.15.99:0/2161026751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009fb0 tx=0x7f8f1c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f1c0038b0 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f1c011e50 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f1c019420 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c1a2670 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c1a5360 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8f1c019a60 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.825+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8f1c024730 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.826+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c069250 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.829+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8f1c043dd0 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.904+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8f2c1a5530 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.908+0000 7f8f23400700 1 -- 172.21.15.99:0/2161026751 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8f1c032030 con 0x7f8f2c081910 2024-03-23T23:32:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 msgr2=0x7f8f2c1a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 --2- 172.21.15.99:0/2161026751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009fb0 tx=0x7f8f1c011bf0 comp rx=0 tx=0).stop 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 shutdown_connections 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 --2- 172.21.15.99:0/2161026751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2c081910 0x7f8f2c1a1ea0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 >> 172.21.15.99:0/2161026751 conn(0x7f8f2c07cf30 msgr2=0x7f8f2c07da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 shutdown_connections 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:02.909+0000 7f8f2be00700 1 -- 172.21.15.99:0/2161026751 wait complete. 2024-03-23T23:32:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-23T23:32:03.785 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:32:03.785 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mgr... 2024-03-23T23:32:03.787 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-23T23:32:03.788 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-23T23:32:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:03 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2161026751' entity='client.admin' 2024-03-23T23:32:04.191 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x 2024-03-23T23:32:04.191 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-23T23:32:04.444 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715.target.wants/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@.service. 2024-03-23T23:32:04.825 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:04 smithi099 systemd[1]: Starting Ceph mgr.x for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:32:05.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:04 smithi099 podman[31931]: 2024-03-23 23:32:04.730873502 +0000 UTC m=+0.032074539 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:32:05.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:04 smithi099 podman[31931]: 2024-03-23 23:32:04.879824764 +0000 UTC m=+0.181025841 container create 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:32:05.299 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T23:32:05.299 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-23T23:32:05.318 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T23:32:05.318 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:32:05.318 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-23T23:32:05.318 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T23:32:05.346 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T23:32:05.346 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-23T23:32:05.365 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T23:32:05.365 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:32:05.365 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-23T23:32:05.366 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-23T23:32:05.366 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr to start... 2024-03-23T23:32:05.366 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr... 2024-03-23T23:32:05.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:05 smithi099 podman[31931]: 2024-03-23 23:32:05.159026183 +0000 UTC m=+0.460227264 container init 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-23T23:32:05.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:05 smithi099 podman[31931]: 2024-03-23 23:32:05.168682823 +0000 UTC m=+0.469883903 container start 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:32:05.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:05 smithi099 bash[31931]: 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 2024-03-23T23:32:05.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:05 smithi099 systemd[1]: Started Ceph mgr.x for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:32:06.078 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:05 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:05.947+0000 7f51d4ebd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T23:32:06.079 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:06 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:06.072+0000 7f51d4ebd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T23:32:06.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:06 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/267769615' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T23:32:06.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:06.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:06.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:32:06.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T23:32:06.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T23:32:06.817 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T23:32:06.825 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T23:32:06.826 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T23:31:55.388101+0000", 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.981+0000 7fe3de600700 1 Processor -- start 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.981+0000 7fe3de600700 1 -- start start 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.982+0000 7fe3de600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.982+0000 7fe3de600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d8107350 con 0x7fe3d8106960 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.982+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:06.827 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.982+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49788/0 (socket says 172.21.15.99:49788) 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.982+0000 7fe3d7400700 1 -- 172.21.15.99:0/345971005 learned_addr learned my addr 172.21.15.99:0/345971005 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.983+0000 7fe3d7400700 1 -- 172.21.15.99:0/345971005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d8107b60 con 0x7fe3d8106960 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.983+0000 7fe3d7400700 1 --2- 172.21.15.99:0/345971005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098a0 tx=0x7fe3c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e111c532a70f9119 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.983+0000 7fe3d6000700 1 -- 172.21.15.99:0/345971005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8004160 con 0x7fe3d8106960 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.983+0000 7fe3d6000700 1 -- 172.21.15.99:0/345971005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3c8004300 con 0x7fe3d8106960 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.983+0000 7fe3d6000700 1 -- 172.21.15.99:0/345971005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8011b50 con 0x7fe3d8106960 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 -- 172.21.15.99:0/345971005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 msgr2=0x7fe3d8106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 --2- 172.21.15.99:0/345971005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098a0 tx=0x7fe3c8009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 -- 172.21.15.99:0/345971005 shutdown_connections 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 --2- 172.21.15.99:0/345971005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d8106d80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 -- 172.21.15.99:0/345971005 >> 172.21.15.99:0/345971005 conn(0x7fe3d8101f20 msgr2=0x7fe3d8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 -- 172.21.15.99:0/345971005 shutdown_connections 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.984+0000 7fe3de600700 1 -- 172.21.15.99:0/345971005 wait complete. 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.985+0000 7fe3de600700 1 Processor -- start 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3de600700 1 -- start start 2024-03-23T23:32:06.828 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3de600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3de600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d810af10 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49794/0 (socket says 172.21.15.99:49794) 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3d7400700 1 -- 172.21.15.99:0/267769615 learned_addr learned my addr 172.21.15.99:0/267769615 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.986+0000 7fe3d7400700 1 -- 172.21.15.99:0/267769615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d819bff0 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3d7400700 1 --2- 172.21.15.99:0/267769615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8011850 tx=0x7fe3c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8003af0 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3c8020070 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d819c280 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3c80195a0 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d819ef70 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.987+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe3c8019c30 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.988+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe3c8011e00 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.988+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d8069250 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:05.991+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe3c8003c90 con 0x7fe3d8106960 2024-03-23T23:32:06.829 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.070+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe3d802dd20 con 0x7fe3d8106960 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.071+0000 7fe3cfe00700 1 -- 172.21.15.99:0/267769615 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe3c801a430 con 0x7fe3d8106960 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 msgr2=0x7fe3d819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 --2- 172.21.15.99:0/267769615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8011850 tx=0x7fe3c8011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 shutdown_connections 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 --2- 172.21.15.99:0/267769615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3d8106960 0x7fe3d819bab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 >> 172.21.15.99:0/267769615 conn(0x7fe3d8101f20 msgr2=0x7fe3d8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 shutdown_connections 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:06.073+0000 7fe3de600700 1 -- 172.21.15.99:0/267769615 wait complete. 2024-03-23T23:32:06.830 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (1/15)... 2024-03-23T23:32:07.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:06 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:06.881+0000 7f51d4ebd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T23:32:07.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.000+0000 7f51d4ebd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T23:32:07.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.176+0000 7f51d4ebd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T23:32:07.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.299+0000 7f51d4ebd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T23:32:07.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.406+0000 7f51d4ebd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T23:32:08.106 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.735+0000 7f51d4ebd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T23:32:08.106 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.873+0000 7f51d4ebd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T23:32:08.106 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:07 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:07.983+0000 7f51d4ebd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T23:32:08.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:08 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:08.106+0000 7f51d4ebd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T23:32:08.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:08 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:08.322+0000 7f51d4ebd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T23:32:09.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:09 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:09.681+0000 7f51d4ebd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T23:32:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:09 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1397247677' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T23:32:10.293 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:09 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:09.870+0000 7f51d4ebd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T23:32:10.293 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:10.292+0000 7f51d4ebd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T23:32:10.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:10.410+0000 7f51d4ebd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T23:32:10.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:10.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:10.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-23T23:32:10.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T23:32:10.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T23:31:55.388101+0000", 2024-03-23T23:32:10.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.463+0000 7f0d14e00700 1 Processor -- start 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.463+0000 7f0d14e00700 1 -- start start 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.463+0000 7f0d14e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.463+0000 7f0d14e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1010a680 con 0x7f0d10109c90 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.464+0000 7f0d0de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.464+0000 7f0d0de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56566/0 (socket says 172.21.15.99:56566) 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.464+0000 7f0d0de00700 1 -- 172.21.15.99:0/3302298877 learned_addr learned my addr 172.21.15.99:0/3302298877 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.464+0000 7f0d0de00700 1 -- 172.21.15.99:0/3302298877 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1010ae90 con 0x7f0d10109c90 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.464+0000 7f0d0de00700 1 --2- 172.21.15.99:0/3302298877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d000098a0 tx=0x7f0d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a08b8199cbaf249 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:10.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.465+0000 7f0d0ca00700 1 -- 172.21.15.99:0/3302298877 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d00004160 con 0x7f0d10109c90 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.465+0000 7f0d0ca00700 1 -- 172.21.15.99:0/3302298877 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d00004300 con 0x7f0d10109c90 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.465+0000 7f0d0ca00700 1 -- 172.21.15.99:0/3302298877 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d00011b50 con 0x7f0d10109c90 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.465+0000 7f0d14e00700 1 -- 172.21.15.99:0/3302298877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 msgr2=0x7f0d1010a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.465+0000 7f0d14e00700 1 --2- 172.21.15.99:0/3302298877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d000098a0 tx=0x7f0d00009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.466+0000 7f0d14e00700 1 -- 172.21.15.99:0/3302298877 shutdown_connections 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.466+0000 7f0d14e00700 1 --2- 172.21.15.99:0/3302298877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d1010a0b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.466+0000 7f0d14e00700 1 -- 172.21.15.99:0/3302298877 >> 172.21.15.99:0/3302298877 conn(0x7f0d1007d050 msgr2=0x7f0d1007d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.466+0000 7f0d14e00700 1 -- 172.21.15.99:0/3302298877 shutdown_connections 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.466+0000 7f0d14e00700 1 -- 172.21.15.99:0/3302298877 wait complete. 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.467+0000 7f0d14e00700 1 Processor -- start 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.467+0000 7f0d14e00700 1 -- start start 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.467+0000 7f0d14e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.467+0000 7f0d14e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1010e240 con 0x7f0d10109c90 2024-03-23T23:32:10.760 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.467+0000 7f0d0de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d0de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56578/0 (socket says 172.21.15.99:56578) 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d0de00700 1 -- 172.21.15.99:0/1397247677 learned_addr learned my addr 172.21.15.99:0/1397247677 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d0de00700 1 -- 172.21.15.99:0/1397247677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d101a2630 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d0de00700 1 --2- 172.21.15.99:0/1397247677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009fb0 tx=0x7f0d00011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d000038b0 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d00011e50 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d101a28c0 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.468+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d00019550 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.469+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d101a55b0 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.469+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0d00019850 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.469+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0d0002dcd0 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.470+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10069250 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.474+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0d00043dd0 con 0x7f0d10109c90 2024-03-23T23:32:10.761 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.562+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0d101a5b00 con 0x7f0d10109c90 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.563+0000 7f0d06a00700 1 -- 172.21.15.99:0/1397247677 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f0d00032030 con 0x7f0d10109c90 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.564+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 msgr2=0x7f0d101a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.564+0000 7f0d14e00700 1 --2- 172.21.15.99:0/1397247677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009fb0 tx=0x7f0d00011bf0 comp rx=0 tx=0).stop 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.565+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 shutdown_connections 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.565+0000 7f0d14e00700 1 --2- 172.21.15.99:0/1397247677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10109c90 0x7f0d101a20f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.565+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 >> 172.21.15.99:0/1397247677 conn(0x7f0d1007d050 msgr2=0x7f0d101076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.565+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 shutdown_connections 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:09.565+0000 7f0d14e00700 1 -- 172.21.15.99:0/1397247677 wait complete. 2024-03-23T23:32:10.762 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (2/15)... 2024-03-23T23:32:11.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:10.667+0000 7f51d4ebd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:32:11.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:10.908+0000 7f51d4ebd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T23:32:11.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:11.279+0000 7f51d4ebd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:32:11.919 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:11.613+0000 7f51d4ebd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T23:32:12.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:11.918+0000 7f51d4ebd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T23:32:13.216 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:12 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:12.950+0000 7f51d4ebd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T23:32:13.797 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:13 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:13.667+0000 7f51d4ebd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T23:32:13.798 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:13 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3294338709' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T23:32:14.078 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:13 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:13.797+0000 7f51d4ebd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T23:32:14.337 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:14.337 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:14.337 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:32:14.338 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T23:32:14.338 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T23:32:14.338 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T23:32:14.338 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:14.078+0000 7f51d4ebd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:14.349 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T23:32:14.350 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T23:32:14.351 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:14.352 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T23:31:55.388101+0000", 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.366+0000 7f8808400700 1 Processor -- start 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.366+0000 7f8808400700 1 -- start start 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8808400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8808400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8800108480 con 0x7f8800107a90 2024-03-23T23:32:14.353 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8805a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8805a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56580/0 (socket says 172.21.15.99:56580) 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8805a00700 1 -- 172.21.15.99:0/1910855909 learned_addr learned my addr 172.21.15.99:0/1910855909 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.367+0000 7f8805a00700 1 -- 172.21.15.99:0/1910855909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800108c90 con 0x7f8800107a90 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.368+0000 7f8805a00700 1 --2- 172.21.15.99:0/1910855909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098a0 tx=0x7f87f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a748dc60a3d7ce2d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.368+0000 7f87ffe00700 1 -- 172.21.15.99:0/1910855909 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87f4004160 con 0x7f8800107a90 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.368+0000 7f87ffe00700 1 -- 172.21.15.99:0/1910855909 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f4004300 con 0x7f8800107a90 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.368+0000 7f87ffe00700 1 -- 172.21.15.99:0/1910855909 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87f4011b50 con 0x7f8800107a90 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 -- 172.21.15.99:0/1910855909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 msgr2=0x7f8800107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 --2- 172.21.15.99:0/1910855909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098a0 tx=0x7f87f4009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 -- 172.21.15.99:0/1910855909 shutdown_connections 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 --2- 172.21.15.99:0/1910855909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f8800107eb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:14.354 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 -- 172.21.15.99:0/1910855909 >> 172.21.15.99:0/1910855909 conn(0x7f8800103050 msgr2=0x7f8800105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 -- 172.21.15.99:0/1910855909 shutdown_connections 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.369+0000 7f8808400700 1 -- 172.21.15.99:0/1910855909 wait complete. 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.370+0000 7f8808400700 1 Processor -- start 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.370+0000 7f8808400700 1 -- start start 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.370+0000 7f8808400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.371+0000 7f8808400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880010c040 con 0x7f8800107a90 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.371+0000 7f8805a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.371+0000 7f8805a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56590/0 (socket says 172.21.15.99:56590) 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.371+0000 7f8805a00700 1 -- 172.21.15.99:0/3294338709 learned_addr learned my addr 172.21.15.99:0/3294338709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.371+0000 7f8805a00700 1 -- 172.21.15.99:0/3294338709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880019c010 con 0x7f8800107a90 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f8805a00700 1 --2- 172.21.15.99:0/3294338709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f4011850 tx=0x7f87f4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:14.355 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87f4003af0 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f4020070 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87f40195a0 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880019c2a0 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880019ef90 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f87f4019be0 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.372+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f87f4011e00 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.373+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800069250 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.377+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f87f4003c90 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.456+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f880019f160 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.456+0000 7f87fe000700 1 -- 172.21.15.99:0/3294338709 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f87f4025460 con 0x7f8800107a90 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 msgr2=0x7f880019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 --2- 172.21.15.99:0/3294338709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f4011850 tx=0x7f87f4011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 shutdown_connections 2024-03-23T23:32:14.356 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 --2- 172.21.15.99:0/3294338709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8800107a90 0x7f880019bad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:14.357 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 >> 172.21.15.99:0/3294338709 conn(0x7f8800103050 msgr2=0x7f8800103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:14.357 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 shutdown_connections 2024-03-23T23:32:14.357 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:13.458+0000 7f8808400700 1 -- 172.21.15.99:0/3294338709 wait complete. 2024-03-23T23:32:14.357 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (3/15)... 2024-03-23T23:32:14.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: Activating manager daemon x 2024-03-23T23:32:14.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: mgrmap e2: x(active, starting, since 0.00972218s) 2024-03-23T23:32:14.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T23:32:14.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T23:32:14.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: Manager daemon x is now available 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' 2024-03-23T23:32:14.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:14 smithi099 ceph-mon[31715]: from='mgr.14100 172.21.15.99:0/1432977930' entity='mgr.x' 2024-03-23T23:32:16.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:16 smithi099 ceph-mon[31715]: mgrmap e3: x(active, since 1.01857s) 2024-03-23T23:32:17.360 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:17 smithi099 ceph-mon[31715]: mgrmap e4: x(active, since 2s) 2024-03-23T23:32:18.353 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:18 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2294470398' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T23:32:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:32:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T23:32:18.360 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:18.361 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T23:32:18.362 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T23:32:18.363 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T23:31:55.388101+0000", 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:18.364 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.000+0000 7efcdf400700 1 Processor -- start 2024-03-23T23:32:18.365 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.000+0000 7efcdf400700 1 -- start start 2024-03-23T23:32:18.369 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.000+0000 7efcdf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.000+0000 7efcdf400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd81098f0 con 0x7efcd810b3b0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.001+0000 7efcdca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.001+0000 7efcdca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56660/0 (socket says 172.21.15.99:56660) 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.001+0000 7efcdca00700 1 -- 172.21.15.99:0/2190389723 learned_addr learned my addr 172.21.15.99:0/2190389723 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.001+0000 7efcdca00700 1 -- 172.21.15.99:0/2190389723 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8109a70 con 0x7efcd810b3b0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.001+0000 7efcdca00700 1 --2- 172.21.15.99:0/2190389723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efccc0098a0 tx=0x7efccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2c2e2fcc6856c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.002+0000 7efcd7400700 1 -- 172.21.15.99:0/2190389723 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efccc004160 con 0x7efcd810b3b0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.002+0000 7efcd7400700 1 -- 172.21.15.99:0/2190389723 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efccc004300 con 0x7efcd810b3b0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.002+0000 7efcd7400700 1 -- 172.21.15.99:0/2190389723 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efccc011b50 con 0x7efcd810b3b0 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 -- 172.21.15.99:0/2190389723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 msgr2=0x7efcd810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 --2- 172.21.15.99:0/2190389723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efccc0098a0 tx=0x7efccc009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 -- 172.21.15.99:0/2190389723 shutdown_connections 2024-03-23T23:32:18.370 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 --2- 172.21.15.99:0/2190389723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd810b7d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 -- 172.21.15.99:0/2190389723 >> 172.21.15.99:0/2190389723 conn(0x7efcd807d0a0 msgr2=0x7efcd807d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 -- 172.21.15.99:0/2190389723 shutdown_connections 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.003+0000 7efcdf400700 1 -- 172.21.15.99:0/2190389723 wait complete. 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.004+0000 7efcdf400700 1 Processor -- start 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.004+0000 7efcdf400700 1 -- start start 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdf400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd810e280 con 0x7efcd810b3b0 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56674/0 (socket says 172.21.15.99:56674) 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdca00700 1 -- 172.21.15.99:0/2294470398 learned_addr learned my addr 172.21.15.99:0/2294470398 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdca00700 1 -- 172.21.15.99:0/2294470398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd819e340 con 0x7efcd810b3b0 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.005+0000 7efcdca00700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fb0 tx=0x7efccc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.006+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efccc0038b0 con 0x7efcd810b3b0 2024-03-23T23:32:18.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.006+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efccc011e50 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.006+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efccc019550 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.006+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd819e5d0 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.006+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd81a12c0 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.007+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd8069250 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.007+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7efccc003a50 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.007+0000 7efcd5600700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 0x7efcc003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.008+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efccc04d060 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.008+0000 7efcd7e00700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 0x7efcc003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.008+0000 7efcd7e00700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 0x7efcc003b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006fd0 tx=0x7efcc4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.014+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efccc04ca80 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.317+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efcd81a1800 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.318+0000 7efcd5600700 1 -- 172.21.15.99:0/2294470398 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7efccc023390 con 0x7efcd810b3b0 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.320+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 msgr2=0x7efcc003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.320+0000 7efcdf400700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 0x7efcc003b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006fd0 tx=0x7efcc4006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 msgr2=0x7efcd819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fb0 tx=0x7efccc011bf0 comp rx=0 tx=0).stop 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 shutdown_connections 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7efcc0038ed0 0x7efcc003b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 --2- 172.21.15.99:0/2294470398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcd810b3b0 0x7efcd819de00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 >> 172.21.15.99:0/2294470398 conn(0x7efcd807d0a0 msgr2=0x7efcd8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 shutdown_connections 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:17.321+0000 7efcdf400700 1 -- 172.21.15.99:0/2294470398 wait complete. 2024-03-23T23:32:18.373 INFO:teuthology.orchestra.run.smithi099.stdout:mgr is available 2024-03-23T23:32:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:19 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2572217009' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T23:32:20.058 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T23:32:20.059 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.031+0000 7f6d46000700 1 Processor -- start 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.031+0000 7f6d46000700 1 -- start start 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.032+0000 7f6d46000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.032+0000 7f6d46000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4010a4d0 con 0x7f6d40109ae0 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.032+0000 7f6d44c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:20.060 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.032+0000 7f6d44c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57126/0 (socket says 172.21.15.99:57126) 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.032+0000 7f6d44c00700 1 -- 172.21.15.99:0/2017594727 learned_addr learned my addr 172.21.15.99:0/2017594727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.033+0000 7f6d44c00700 1 -- 172.21.15.99:0/2017594727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4010ac90 con 0x7f6d40109ae0 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.033+0000 7f6d44c00700 1 --2- 172.21.15.99:0/2017594727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d340098a0 tx=0x7f6d34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06da26359cd418c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.034+0000 7f6d3f400700 1 -- 172.21.15.99:0/2017594727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d34004160 con 0x7f6d40109ae0 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.034+0000 7f6d3f400700 1 -- 172.21.15.99:0/2017594727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d34004300 con 0x7f6d40109ae0 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.034+0000 7f6d3f400700 1 -- 172.21.15.99:0/2017594727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d34011b50 con 0x7f6d40109ae0 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.034+0000 7f6d46000700 1 -- 172.21.15.99:0/2017594727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 msgr2=0x7f6d40109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.034+0000 7f6d46000700 1 --2- 172.21.15.99:0/2017594727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d340098a0 tx=0x7f6d34009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.035+0000 7f6d46000700 1 -- 172.21.15.99:0/2017594727 shutdown_connections 2024-03-23T23:32:20.061 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.035+0000 7f6d46000700 1 --2- 172.21.15.99:0/2017594727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d40109f00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.035+0000 7f6d46000700 1 -- 172.21.15.99:0/2017594727 >> 172.21.15.99:0/2017594727 conn(0x7f6d4007cf30 msgr2=0x7f6d4007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.035+0000 7f6d46000700 1 -- 172.21.15.99:0/2017594727 shutdown_connections 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.035+0000 7f6d46000700 1 -- 172.21.15.99:0/2017594727 wait complete. 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.036+0000 7f6d46000700 1 Processor -- start 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.036+0000 7f6d46000700 1 -- start start 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.036+0000 7f6d46000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.036+0000 7f6d46000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4010e040 con 0x7f6d40109ae0 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.037+0000 7f6d44c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.037+0000 7f6d44c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57130/0 (socket says 172.21.15.99:57130) 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.037+0000 7f6d44c00700 1 -- 172.21.15.99:0/2572217009 learned_addr learned my addr 172.21.15.99:0/2572217009 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.037+0000 7f6d44c00700 1 -- 172.21.15.99:0/2572217009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4019e020 con 0x7f6d40109ae0 2024-03-23T23:32:20.062 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.037+0000 7f6d44c00700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6d34011850 tx=0x7f6d34011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.038+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d34003af0 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.038+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d34020070 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.038+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d340195a0 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.038+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4019e2b0 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.038+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401a0fa0 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.039+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f6d34019850 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.039+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40069250 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.040+0000 7f6d3d600700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 0x7f6d2803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.040+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d34011e00 con 0x7f6d40109ae0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.040+0000 7f6d3fe00700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 0x7f6d2803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:20.063 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.040+0000 7f6d3fe00700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 0x7f6d2803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006fd0 tx=0x7f6d2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.045+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d34003c90 con 0x7f6d40109ae0 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.266+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6d401a2760 con 0x7f6d40109ae0 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.269+0000 7f6d3d600700 1 -- 172.21.15.99:0/2572217009 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f6d3404cc20 con 0x7f6d40109ae0 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 msgr2=0x7f6d2803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 0x7f6d2803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c006fd0 tx=0x7f6d2c006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 msgr2=0x7f6d4019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6d34011850 tx=0x7f6d34011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 shutdown_connections 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f6d28038f20 0x7f6d2803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 --2- 172.21.15.99:0/2572217009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40109ae0 0x7f6d4019dae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.271+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 >> 172.21.15.99:0/2572217009 conn(0x7f6d4007cf30 msgr2=0x7f6d40107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:20.064 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.272+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 shutdown_connections 2024-03-23T23:32:20.065 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:19.272+0000 7f6d46000700 1 -- 172.21.15.99:0/2572217009 wait complete. 2024-03-23T23:32:20.065 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling cephadm module... 2024-03-23T23:32:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:21 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3872006963' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-23T23:32:21.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: ignoring --setuser ceph since I am not root 2024-03-23T23:32:21.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: ignoring --setgroup ceph since I am not root 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.709+0000 7f08e2c00700 1 Processor -- start 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.709+0000 7f08e2c00700 1 -- start start 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08e2c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08e2c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc07f270 con 0x7f08dc07e880 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57134/0 (socket says 172.21.15.99:57134) 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08dbe00700 1 -- 172.21.15.99:0/857737729 learned_addr learned my addr 172.21.15.99:0/857737729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.710+0000 7f08dbe00700 1 -- 172.21.15.99:0/857737729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc07fad0 con 0x7f08dc07e880 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.711+0000 7f08dbe00700 1 --2- 172.21.15.99:0/857737729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098a0 tx=0x7f08cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5deff36af11b2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.711+0000 7f08db400700 1 -- 172.21.15.99:0/857737729 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08cc004160 con 0x7f08dc07e880 2024-03-23T23:32:21.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.711+0000 7f08db400700 1 -- 172.21.15.99:0/857737729 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08cc004300 con 0x7f08dc07e880 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.711+0000 7f08db400700 1 -- 172.21.15.99:0/857737729 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08cc011b50 con 0x7f08dc07e880 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 -- 172.21.15.99:0/857737729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 msgr2=0x7f08dc07eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 --2- 172.21.15.99:0/857737729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098a0 tx=0x7f08cc009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 -- 172.21.15.99:0/857737729 shutdown_connections 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 --2- 172.21.15.99:0/857737729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc07eca0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 -- 172.21.15.99:0/857737729 >> 172.21.15.99:0/857737729 conn(0x7f08dc07d0d0 msgr2=0x7f08dc07d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 -- 172.21.15.99:0/857737729 shutdown_connections 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.712+0000 7f08e2c00700 1 -- 172.21.15.99:0/857737729 wait complete. 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.713+0000 7f08e2c00700 1 Processor -- start 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.713+0000 7f08e2c00700 1 -- start start 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.713+0000 7f08e2c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.713+0000 7f08e2c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc080e90 con 0x7f08dc07e880 2024-03-23T23:32:21.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.714+0000 7f08dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.714+0000 7f08dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57142/0 (socket says 172.21.15.99:57142) 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.714+0000 7f08dbe00700 1 -- 172.21.15.99:0/3872006963 learned_addr learned my addr 172.21.15.99:0/3872006963 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.714+0000 7f08dbe00700 1 -- 172.21.15.99:0/3872006963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc1a2550 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.714+0000 7f08dbe00700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08cc011b60 tx=0x7f08cc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.715+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08cc003b20 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.715+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08cc003cc0 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.715+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc1a27e0 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.715+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08cc024840 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.715+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1a54d0 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.716+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f08cc022020 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.716+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc0637c0 con 0x7f08dc07e880 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.717+0000 7f08d9600700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 0x7f08bc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:21.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.717+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f08cc011e00 con 0x7f08dc07e880 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.717+0000 7f08d3400700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 0x7f08bc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.717+0000 7f08d3400700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 0x7f08bc03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.722+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08cc04c8e0 con 0x7f08dc07e880 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:20.999+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f08dc0008d0 con 0x7f08dc07e880 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.052+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f08cc00b3e0 con 0x7f08dc07e880 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.056+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 msgr2=0x7f08bc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.056+0000 7f08e2c00700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 0x7f08bc03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.056+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 msgr2=0x7f08dc1a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.056+0000 7f08e2c00700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08cc011b60 tx=0x7f08cc011c40 comp rx=0 tx=0).stop 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.056+0000 7f08d9600700 1 -- 172.21.15.99:0/3872006963 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f08cc01a7b0 con 0x7f08dc07e880 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.057+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 shutdown_connections 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.057+0000 7f08e2c00700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f08bc038f20 0x7f08bc03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.057+0000 7f08e2c00700 1 --2- 172.21.15.99:0/3872006963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08dc07e880 0x7f08dc1a2010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:21.815 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.057+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 >> 172.21.15.99:0/3872006963 conn(0x7f08dc07d0d0 msgr2=0x7f08dc198cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:21.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.058+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 shutdown_connections 2024-03-23T23:32:21.816 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:21.058+0000 7f08e2c00700 1 -- 172.21.15.99:0/3872006963 wait complete. 2024-03-23T23:32:22.061 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:21.832+0000 7f4209e76200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T23:32:22.061 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:21.960+0000 7f4209e76200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T23:32:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:22 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3872006963' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-23T23:32:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:22 smithi099 ceph-mon[31715]: mgrmap e5: x(active, since 6s) 2024-03-23T23:32:23.061 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:22.773+0000 7f4209e76200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T23:32:23.062 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:22.896+0000 7f4209e76200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T23:32:23.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/759752677' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T23:32:23.313 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.076+0000 7f4209e76200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T23:32:23.313 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.202+0000 7f4209e76200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T23:32:23.313 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.312+0000 7f4209e76200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T23:32:23.553 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.454+0000 7ff707e00700 1 Processor -- start 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.454+0000 7ff707e00700 1 -- start start 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.455+0000 7ff707e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.455+0000 7ff707e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70807be50 con 0x7ff7081083d0 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.455+0000 7ff706a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.455+0000 7ff706a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57182/0 (socket says 172.21.15.99:57182) 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.455+0000 7ff706a00700 1 -- 172.21.15.99:0/1432082026 learned_addr learned my addr 172.21.15.99:0/1432082026 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.456+0000 7ff706a00700 1 -- 172.21.15.99:0/1432082026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70810ad00 con 0x7ff7081083d0 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.456+0000 7ff706a00700 1 --2- 172.21.15.99:0/1432082026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80098a0 tx=0x7ff6f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d984c8d5d32d900 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:23.554 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.456+0000 7ff705600700 1 -- 172.21.15.99:0/1432082026 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6f8004160 con 0x7ff7081083d0 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.456+0000 7ff705600700 1 -- 172.21.15.99:0/1432082026 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6f8004300 con 0x7ff7081083d0 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.456+0000 7ff705600700 1 -- 172.21.15.99:0/1432082026 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6f8011b50 con 0x7ff7081083d0 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 -- 172.21.15.99:0/1432082026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 msgr2=0x7ff70810a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 --2- 172.21.15.99:0/1432082026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80098a0 tx=0x7ff6f8009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 -- 172.21.15.99:0/1432082026 shutdown_connections 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 --2- 172.21.15.99:0/1432082026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70810a7c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 -- 172.21.15.99:0/1432082026 >> 172.21.15.99:0/1432082026 conn(0x7ff708101d60 msgr2=0x7ff7081041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 -- 172.21.15.99:0/1432082026 shutdown_connections 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.457+0000 7ff707e00700 1 -- 172.21.15.99:0/1432082026 wait complete. 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.458+0000 7ff707e00700 1 Processor -- start 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.458+0000 7ff707e00700 1 -- start start 2024-03-23T23:32:23.555 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff707e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff707e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70810bcc0 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff706a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff706a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57186/0 (socket says 172.21.15.99:57186) 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff706a00700 1 -- 172.21.15.99:0/759752677 learned_addr learned my addr 172.21.15.99:0/759752677 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.459+0000 7ff706a00700 1 -- 172.21.15.99:0/759752677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70819e060 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff706a00700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8011b60 tx=0x7ff6f8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6f8003970 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6f8003b10 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70819e2f0 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6f8024610 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.460+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7081a0fe0 con 0x7ff7081083d0 2024-03-23T23:32:23.556 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.461+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7ff6f80248c0 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.461+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff708069250 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.462+0000 7ff6ff400700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 0x7ff6ec03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.462+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff6f8011e00 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.462+0000 7ff706000700 1 -- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 msgr2=0x7ff6ec03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.462+0000 7ff706000700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 0x7ff6ec03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.467+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6f800b3e0 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.663+0000 7ff706000700 1 -- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 msgr2=0x7ff6ec03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.663+0000 7ff706000700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 0x7ff6ec03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.748+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff7081a11b0 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.749+0000 7ff6ff400700 1 -- 172.21.15.99:0/759752677 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff7081a11b0 con 0x7ff7081083d0 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 msgr2=0x7ff6ec03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:32:23.557 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 0x7ff6ec03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 msgr2=0x7ff70819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8011b60 tx=0x7ff6f8011c40 comp rx=0 tx=0).stop 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 shutdown_connections 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7ff6ec038f70 0x7ff6ec03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 --2- 172.21.15.99:0/759752677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7081083d0 0x7ff70819db20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.752+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 >> 172.21.15.99:0/759752677 conn(0x7ff708101d60 msgr2=0x7ff708102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.753+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 shutdown_connections 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:22.753+0000 7ff707e00700 1 -- 172.21.15.99:0/759752677 wait complete. 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-03-23T23:32:23.558 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 5... 2024-03-23T23:32:23.908 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.651+0000 7f4209e76200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T23:32:23.908 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.794+0000 7f4209e76200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T23:32:23.908 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:23.907+0000 7f4209e76200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T23:32:24.191 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:24.038+0000 7f4209e76200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T23:32:24.544 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:24.265+0000 7f4209e76200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T23:32:25.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:25.581+0000 7f4209e76200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T23:32:25.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:25.754+0000 7f4209e76200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T23:32:26.538 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:26.167+0000 7f4209e76200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T23:32:26.538 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:26.284+0000 7f4209e76200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T23:32:26.539 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:26.538+0000 7f4209e76200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:32:26.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:26.780+0000 7f4209e76200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T23:32:27.466 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:27.150+0000 7f4209e76200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:32:28.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:27.466+0000 7f4209e76200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T23:32:28.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:27.765+0000 7f4209e76200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T23:32:29.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:28.774+0000 7f4209e76200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T23:32:29.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:29.466+0000 7f4209e76200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T23:32:29.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:29.594+0000 7f4209e76200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T23:32:30.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:29.870+0000 7f4209e76200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T23:32:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: Active manager daemon x restarted 2024-03-23T23:32:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: Activating manager daemon x 2024-03-23T23:32:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: osdmap e2: 0 total, 0 up, 0 in 2024-03-23T23:32:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: mgrmap e6: x(active, starting, since 0.00902663s) 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: Manager daemon x is now available 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:30.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:29 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: Found migration_current of "None". Setting to last migration. 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T23:32:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T23:32:31.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:31.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:31.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:30 smithi099 ceph-mon[31715]: mgrmap e7: x(active, since 1.01791s) 2024-03-23T23:32:31.745 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:32:31.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-23T23:32:31.746 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.222+0000 7f7a7d000700 1 Processor -- start 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.222+0000 7f7a7d000700 1 -- start start 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.222+0000 7f7a7d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.222+0000 7f7a7d000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7807be50 con 0x7f7a781085a0 2024-03-23T23:32:31.747 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.223+0000 7f7a75e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:31.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.223+0000 7f7a75e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57198/0 (socket says 172.21.15.99:57198) 2024-03-23T23:32:31.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.223+0000 7f7a75e00700 1 -- 172.21.15.99:0/3601979620 learned_addr learned my addr 172.21.15.99:0/3601979620 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:31.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.223+0000 7f7a75e00700 1 -- 172.21.15.99:0/3601979620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7810aed0 con 0x7f7a781085a0 2024-03-23T23:32:31.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.224+0000 7f7a75e00700 1 --2- 172.21.15.99:0/3601979620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a680098a0 tx=0x7f7a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae4a3df85f48368 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:31.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.224+0000 7f7a74a00700 1 -- 172.21.15.99:0/3601979620 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a68004160 con 0x7f7a781085a0 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.224+0000 7f7a74a00700 1 -- 172.21.15.99:0/3601979620 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a68004300 con 0x7f7a781085a0 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.224+0000 7f7a74a00700 1 -- 172.21.15.99:0/3601979620 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a68011b50 con 0x7f7a781085a0 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 -- 172.21.15.99:0/3601979620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 msgr2=0x7f7a7810a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 --2- 172.21.15.99:0/3601979620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a680098a0 tx=0x7f7a68009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 -- 172.21.15.99:0/3601979620 shutdown_connections 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 --2- 172.21.15.99:0/3601979620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a7810a990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 -- 172.21.15.99:0/3601979620 >> 172.21.15.99:0/3601979620 conn(0x7f7a78101f50 msgr2=0x7f7a78104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 -- 172.21.15.99:0/3601979620 shutdown_connections 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.225+0000 7f7a7d000700 1 -- 172.21.15.99:0/3601979620 wait complete. 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.226+0000 7f7a7d000700 1 Processor -- start 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.226+0000 7f7a7d000700 1 -- start start 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a7d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a7d000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7810be90 con 0x7f7a781085a0 2024-03-23T23:32:31.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a75e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a75e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57210/0 (socket says 172.21.15.99:57210) 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a75e00700 1 -- 172.21.15.99:0/44680749 learned_addr learned my addr 172.21.15.99:0/44680749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.227+0000 7f7a75e00700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78108360 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.228+0000 7f7a75e00700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011b60 tx=0x7f7a68011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.228+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a68003970 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.228+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a68003b10 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.228+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a78106550 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.228+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a68024610 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.229+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78106a90 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.229+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7a680248c0 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.230+0000 7f7a6ea00700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.230+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a78069250 con 0x7f7a5c038f70 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.230+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7a68011e00 con 0x7f7a781085a0 2024-03-23T23:32:31.756 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.230+0000 7f7a75400700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.230+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.431+0000 7f7a75400700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.431+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.832+0000 7f7a75400700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:24.832+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:25.633+0000 7f7a75400700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:25.633+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:27.235+0000 7f7a75400700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/3938208331 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:27.235+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:29.880+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7f7a68019a20 con 0x7f7a781085a0 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:29.880+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 msgr2=0x7f7a5c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:29.880+0000 7f7a6ea00700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.757 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.885+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7f7a680424b0 con 0x7f7a781085a0 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.886+0000 7f7a6ea00700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 0x7f7a5c03ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.886+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a78069250 con 0x7f7a5c03c670 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.887+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 0x7f7a5c03ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.890+0000 7f7a75400700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 0x7f7a5c03ea60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7a64003a10 tx=0x7f7a640092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.891+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f7a78069250 con 0x7f7a5c03c670 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.899+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7a781031f0 con 0x7f7a5c03c670 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.899+0000 7f7a6ea00700 1 -- 172.21.15.99:0/44680749 <== mgr.14120 v2:172.21.15.99:6800/2749486918 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7a781031f0 con 0x7f7a5c03c670 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 msgr2=0x7f7a5c03ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 0x7f7a5c03ea60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7a64003a10 tx=0x7f7a640092b0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 msgr2=0x7f7a78107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a68011b60 tx=0x7f7a68011c40 comp rx=0 tx=0).stop 2024-03-23T23:32:31.758 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 shutdown_connections 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f7a5c03c670 0x7f7a5c03ea60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a781085a0 0x7f7a78107e20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 --2- 172.21.15.99:0/44680749 >> [v2:172.21.15.99:6800/3938208331,v1:172.21.15.99:6801/3938208331] conn(0x7f7a5c038f70 0x7f7a5c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 >> 172.21.15.99:0/44680749 conn(0x7f7a78101f50 msgr2=0x7f7a78102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 shutdown_connections 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:30.900+0000 7f7a7d000700 1 -- 172.21.15.99:0/44680749 wait complete. 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 5 is available 2024-03-23T23:32:31.759 INFO:teuthology.orchestra.run.smithi099.stdout:Setting orchestrator backend to cephadm... 2024-03-23T23:32:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T23:32:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: [23/Mar/2024:23:32:30] ENGINE Bus STARTING 2024-03-23T23:32:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T23:32:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: [23/Mar/2024:23:32:30] ENGINE Serving on http://172.21.15.99:8765 2024-03-23T23:32:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: [23/Mar/2024:23:32:31] ENGINE Serving on https://172.21.15.99:7150 2024-03-23T23:32:32.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: [23/Mar/2024:23:32:31] ENGINE Bus STARTED 2024-03-23T23:32:32.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:31 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:33.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:32 smithi099 ceph-mon[31715]: mgrmap e8: x(active, since 2s) 2024-03-23T23:32:33.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:32 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:33.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:32 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.414+0000 7fc0a9000700 1 Processor -- start 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.414+0000 7fc0a9000700 1 -- start start 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.414+0000 7fc0a9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.414+0000 7fc0a9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a410a680 con 0x7fc0a4109c90 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.415+0000 7fc0a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.415+0000 7fc0a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59726/0 (socket says 172.21.15.99:59726) 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.415+0000 7fc0a1e00700 1 -- 172.21.15.99:0/3032646393 learned_addr learned my addr 172.21.15.99:0/3032646393 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.415+0000 7fc0a1e00700 1 -- 172.21.15.99:0/3032646393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a410ae90 con 0x7fc0a4109c90 2024-03-23T23:32:33.478 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.416+0000 7fc0a1e00700 1 --2- 172.21.15.99:0/3032646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098a0 tx=0x7fc094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc414b065446d1ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:33.482 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.416+0000 7fc0a0a00700 1 -- 172.21.15.99:0/3032646393 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc094004160 con 0x7fc0a4109c90 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.416+0000 7fc0a0a00700 1 -- 172.21.15.99:0/3032646393 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc094004300 con 0x7fc0a4109c90 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.416+0000 7fc0a0a00700 1 -- 172.21.15.99:0/3032646393 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc094011b50 con 0x7fc0a4109c90 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 -- 172.21.15.99:0/3032646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 msgr2=0x7fc0a410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3032646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098a0 tx=0x7fc094009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 -- 172.21.15.99:0/3032646393 shutdown_connections 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3032646393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a410a0b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 -- 172.21.15.99:0/3032646393 >> 172.21.15.99:0/3032646393 conn(0x7fc0a407d0d0 msgr2=0x7fc0a407d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 -- 172.21.15.99:0/3032646393 shutdown_connections 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.417+0000 7fc0a9000700 1 -- 172.21.15.99:0/3032646393 wait complete. 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.418+0000 7fc0a9000700 1 Processor -- start 2024-03-23T23:32:33.483 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a9000700 1 -- start start 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a410e240 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59740/0 (socket says 172.21.15.99:59740) 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.419+0000 7fc0a1e00700 1 -- 172.21.15.99:0/3429817224 learned_addr learned my addr 172.21.15.99:0/3429817224 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc0a1e00700 1 -- 172.21.15.99:0/3429817224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a419e1a0 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc0a1e00700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc094003b20 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc094003cc0 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc094024910 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a419e430 con 0x7fc0a4109c90 2024-03-23T23:32:33.484 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.420+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a41a1120 con 0x7fc0a4109c90 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.421+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fc094022020 con 0x7fc0a4109c90 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.421+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a40637c0 con 0x7fc0a4109c90 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.422+0000 7fc09aa00700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 0x7fc08803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.422+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc094011e00 con 0x7fc0a4109c90 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.422+0000 7fc0a1400700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 0x7fc08803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.423+0000 7fc0a1400700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 0x7fc08803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc090006fd0 tx=0x7fc090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.428+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc094016260 con 0x7fc0a4109c90 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.663+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0a4107d30 con 0x7fc088038f70 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.677+0000 7fc09aa00700 1 -- 172.21.15.99:0/3429817224 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc0a4107d30 con 0x7fc088038f70 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 msgr2=0x7fc08803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 0x7fc08803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc090006fd0 tx=0x7fc090006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:33.485 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 msgr2=0x7fc0a419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc094000c00 tx=0x7fc094011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 shutdown_connections 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fc088038f70 0x7fc08803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 --2- 172.21.15.99:0/3429817224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0a4109c90 0x7fc0a419dc60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 >> 172.21.15.99:0/3429817224 conn(0x7fc0a407d0d0 msgr2=0x7fc0a4107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.680+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 shutdown_connections 2024-03-23T23:32:33.486 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:32.681+0000 7fc0a9000700 1 -- 172.21.15.99:0/3429817224 wait complete. 2024-03-23T23:32:34.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:33 smithi099 ceph-mon[31715]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:35.294 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-23T23:32:35.295 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.226+0000 7fd68cc00700 1 Processor -- start 2024-03-23T23:32:35.295 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.226+0000 7fd68cc00700 1 -- start start 2024-03-23T23:32:35.295 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.226+0000 7fd68cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:35.295 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.226+0000 7fd68cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68807c8f0 con 0x7fd68807dec0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.227+0000 7fd685e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.227+0000 7fd685e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59756/0 (socket says 172.21.15.99:59756) 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.227+0000 7fd685e00700 1 -- 172.21.15.99:0/2113503188 learned_addr learned my addr 172.21.15.99:0/2113503188 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.227+0000 7fd685e00700 1 -- 172.21.15.99:0/2113503188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68807ca70 con 0x7fd68807dec0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.228+0000 7fd685e00700 1 --2- 172.21.15.99:0/2113503188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd6780098a0 tx=0x7fd678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef14d189434173f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.228+0000 7fd684a00700 1 -- 172.21.15.99:0/2113503188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd678004160 con 0x7fd68807dec0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.228+0000 7fd684a00700 1 -- 172.21.15.99:0/2113503188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd678004300 con 0x7fd68807dec0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.228+0000 7fd684a00700 1 -- 172.21.15.99:0/2113503188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd678011b50 con 0x7fd68807dec0 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 -- 172.21.15.99:0/2113503188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 msgr2=0x7fd68807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 --2- 172.21.15.99:0/2113503188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd6780098a0 tx=0x7fd678009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:35.301 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 -- 172.21.15.99:0/2113503188 shutdown_connections 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 --2- 172.21.15.99:0/2113503188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68807c320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 -- 172.21.15.99:0/2113503188 >> 172.21.15.99:0/2113503188 conn(0x7fd688103020 msgr2=0x7fd688105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 -- 172.21.15.99:0/2113503188 shutdown_connections 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.229+0000 7fd68cc00700 1 -- 172.21.15.99:0/2113503188 wait complete. 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.230+0000 7fd68cc00700 1 Processor -- start 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.230+0000 7fd68cc00700 1 -- start start 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.230+0000 7fd68cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.230+0000 7fd68cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68810e090 con 0x7fd68807dec0 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.231+0000 7fd685e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.231+0000 7fd685e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59770/0 (socket says 172.21.15.99:59770) 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.231+0000 7fd685e00700 1 -- 172.21.15.99:0/3017000087 learned_addr learned my addr 172.21.15.99:0/3017000087 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.231+0000 7fd685e00700 1 -- 172.21.15.99:0/3017000087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68819e1f0 con 0x7fd68807dec0 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.231+0000 7fd685e00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd678009fb0 tx=0x7fd678011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.232+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6780038b0 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.232+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd678011e50 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.232+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd678019550 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.232+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68819e480 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.232+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6881a1170 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.233+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd678003a50 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.233+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd688069250 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.233+0000 7fd67ea00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 0x7fd66c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.233+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd67804d110 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.234+0000 7fd685400700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 0x7fd66c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.234+0000 7fd685400700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 0x7fd66c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd674006fd0 tx=0x7fd674006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.239+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd67804ca60 con 0x7fd68807dec0 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.463+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd688104330 con 0x7fd66c038f20 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.464+0000 7fd67ea00700 1 -- 172.21.15.99:0/3017000087 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd688104330 con 0x7fd66c038f20 2024-03-23T23:32:35.303 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 msgr2=0x7fd66c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 0x7fd66c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd674006fd0 tx=0x7fd674006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 msgr2=0x7fd68819dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd678009fb0 tx=0x7fd678011bf0 comp rx=0 tx=0).stop 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 shutdown_connections 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7fd66c038f20 0x7fd66c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 --2- 172.21.15.99:0/3017000087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd68807dec0 0x7fd68819dcb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 >> 172.21.15.99:0/3017000087 conn(0x7fd688103020 msgr2=0x7fd688103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.467+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 shutdown_connections 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:34.468+0000 7fd68cc00700 1 -- 172.21.15.99:0/3017000087 wait complete. 2024-03-23T23:32:35.304 INFO:teuthology.orchestra.run.smithi099.stdout:Generating ssh key... 2024-03-23T23:32:35.517 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:35 smithi099 ceph-mon[31715]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:37.112 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: Generating public/private rsa key pair. 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: Your identification has been saved in /tmp/tmprjnevjki/key. 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: Your public key has been saved in /tmp/tmprjnevjki/key.pub. 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: The key fingerprint is: 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: SHA256:hejVYDQ3CAQXnIaJshO3Wz7wjPcWVezcTumF8ilijU8 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: The key's randomart image is: 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: +---[RSA 3072]----+ 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | ..*===oo | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: |o o o.+o.*o. | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | = . .. o+o. o | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: |o o .. ...+ = . | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | . O ..So * o | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | o * . + E = | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | . o o + . | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | o . | 2024-03-23T23:32:37.113 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: | . | 2024-03-23T23:32:37.114 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: +----[SHA256]-----+ 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.035+0000 7fecc2200700 1 Processor -- start 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.035+0000 7fecc2200700 1 -- start start 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.036+0000 7fecc2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.036+0000 7fecc2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc10a680 con 0x7fecbc109c90 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.036+0000 7fecbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:37.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.036+0000 7fecbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59778/0 (socket says 172.21.15.99:59778) 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.036+0000 7fecbb400700 1 -- 172.21.15.99:0/3696618856 learned_addr learned my addr 172.21.15.99:0/3696618856 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.037+0000 7fecbb400700 1 -- 172.21.15.99:0/3696618856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc10ae90 con 0x7fecbc109c90 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.037+0000 7fecbb400700 1 --2- 172.21.15.99:0/3696618856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098a0 tx=0x7fecac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576a4b9ea96d993a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.037+0000 7fecbaa00700 1 -- 172.21.15.99:0/3696618856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac004160 con 0x7fecbc109c90 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.037+0000 7fecbaa00700 1 -- 172.21.15.99:0/3696618856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecac004300 con 0x7fecbc109c90 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.037+0000 7fecbaa00700 1 -- 172.21.15.99:0/3696618856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac011b50 con 0x7fecbc109c90 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 -- 172.21.15.99:0/3696618856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 msgr2=0x7fecbc10a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 --2- 172.21.15.99:0/3696618856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098a0 tx=0x7fecac009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:37.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 -- 172.21.15.99:0/3696618856 shutdown_connections 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 --2- 172.21.15.99:0/3696618856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc10a0b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 -- 172.21.15.99:0/3696618856 >> 172.21.15.99:0/3696618856 conn(0x7fecbc07d060 msgr2=0x7fecbc07d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 -- 172.21.15.99:0/3696618856 shutdown_connections 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.038+0000 7fecc2200700 1 -- 172.21.15.99:0/3696618856 wait complete. 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.039+0000 7fecc2200700 1 Processor -- start 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecc2200700 1 -- start start 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecc2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecc2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc10e240 con 0x7fecbc109c90 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59794/0 (socket says 172.21.15.99:59794) 2024-03-23T23:32:37.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecbb400700 1 -- 172.21.15.99:0/3024637504 learned_addr learned my addr 172.21.15.99:0/3024637504 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.040+0000 7fecbb400700 1 -- 172.21.15.99:0/3024637504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc19e220 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecbb400700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecac011850 tx=0x7fecac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac003af0 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecac020070 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc19e4b0 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc1a1190 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.041+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac0195a0 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.042+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fecac01a430 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.042+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc069250 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.043+0000 7fecb8c00700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 0x7feca003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.043+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fecac011e00 con 0x7fecbc109c90 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.043+0000 7fecb3e00700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 0x7feca003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.044+0000 7fecb3e00700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 0x7feca003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feca4006fd0 tx=0x7feca4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:37.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.048+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fecac021270 con 0x7fecbc109c90 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.269+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fecbc107df0 con 0x7feca0038f20 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.786+0000 7fecb8c00700 1 -- 172.21.15.99:0/3024637504 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fecbc107df0 con 0x7feca0038f20 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 msgr2=0x7feca003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 0x7feca003b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feca4006fd0 tx=0x7feca4006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 msgr2=0x7fecbc19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecac011850 tx=0x7fecac011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 shutdown_connections 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7feca0038f20 0x7feca003b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 --2- 172.21.15.99:0/3024637504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecbc109c90 0x7fecbc19dce0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 >> 172.21.15.99:0/3024637504 conn(0x7fecbc07d060 msgr2=0x7fecbc1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 shutdown_connections 2024-03-23T23:32:37.608 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:36.789+0000 7fecc2200700 1 -- 172.21.15.99:0/3024637504 wait complete. 2024-03-23T23:32:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:37 smithi099 ceph-mon[31715]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:37 smithi099 ceph-mon[31715]: Generating ssh key... 2024-03-23T23:32:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:37 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:37 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:39.242 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:32:39.243 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.246+0000 7f850ee00700 1 Processor -- start 2024-03-23T23:32:39.243 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.246+0000 7f850ee00700 1 -- start start 2024-03-23T23:32:39.243 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:39.243 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850810a4d0 con 0x7f8508109ae0 2024-03-23T23:32:39.244 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:39.244 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59804/0 (socket says 172.21.15.99:59804) 2024-03-23T23:32:39.244 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850da00700 1 -- 172.21.15.99:0/31536538 learned_addr learned my addr 172.21.15.99:0/31536538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:39.244 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.247+0000 7f850da00700 1 -- 172.21.15.99:0/31536538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850810ace0 con 0x7f8508109ae0 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.248+0000 7f850da00700 1 --2- 172.21.15.99:0/31536538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0098a0 tx=0x7f84fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d6a5670eb3572e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.248+0000 7f8507e00700 1 -- 172.21.15.99:0/31536538 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84fc004160 con 0x7f8508109ae0 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.248+0000 7f8507e00700 1 -- 172.21.15.99:0/31536538 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84fc004300 con 0x7f8508109ae0 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.248+0000 7f8507e00700 1 -- 172.21.15.99:0/31536538 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84fc011b50 con 0x7f8508109ae0 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 -- 172.21.15.99:0/31536538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 msgr2=0x7f8508109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:39.245 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 --2- 172.21.15.99:0/31536538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0098a0 tx=0x7f84fc009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:39.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 -- 172.21.15.99:0/31536538 shutdown_connections 2024-03-23T23:32:39.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 --2- 172.21.15.99:0/31536538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f8508109f00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:39.246 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 -- 172.21.15.99:0/31536538 >> 172.21.15.99:0/31536538 conn(0x7f850807ce80 msgr2=0x7f850807d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:39.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 -- 172.21.15.99:0/31536538 shutdown_connections 2024-03-23T23:32:39.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.249+0000 7f850ee00700 1 -- 172.21.15.99:0/31536538 wait complete. 2024-03-23T23:32:39.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.250+0000 7f850ee00700 1 Processor -- start 2024-03-23T23:32:39.247 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.250+0000 7f850ee00700 1 -- start start 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.250+0000 7f850ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.250+0000 7f850ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850810e090 con 0x7f8508109ae0 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f850da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f850da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59816/0 (socket says 172.21.15.99:59816) 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f850da00700 1 -- 172.21.15.99:0/286767933 learned_addr learned my addr 172.21.15.99:0/286767933 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:39.248 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f850da00700 1 -- 172.21.15.99:0/286767933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850819e040 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f850da00700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84fc011850 tx=0x7f84fc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.251+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84fc003af0 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.252+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84fc020070 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.252+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84fc0195a0 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.252+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850819e2d0 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.252+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85081a0fc0 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.253+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508069250 con 0x7f8508109ae0 2024-03-23T23:32:39.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.253+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f84fc019850 con 0x7f8508109ae0 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.253+0000 7f8506000700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 0x7f84f003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.253+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f84fc011e00 con 0x7f8508109ae0 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.254+0000 7f850d000700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 0x7f84f003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.254+0000 7f850d000700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 0x7f84f003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.259+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84fc019b40 con 0x7f8508109ae0 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.477+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8508107d30 con 0x7f84f0038f70 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.478+0000 7f8506000700 1 -- 172.21.15.99:0/286767933 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8508107d30 con 0x7f84f0038f70 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 msgr2=0x7f84f003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 0x7f84f003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 msgr2=0x7f850819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84fc011850 tx=0x7f84fc011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 shutdown_connections 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f84f0038f70 0x7f84f003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 --2- 172.21.15.99:0/286767933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8508109ae0 0x7f850819db00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:39.253 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 >> 172.21.15.99:0/286767933 conn(0x7f850807ce80 msgr2=0x7f8508107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:39.253 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 shutdown_connections 2024-03-23T23:32:39.253 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:38.481+0000 7f850ee00700 1 -- 172.21.15.99:0/286767933 wait complete. 2024-03-23T23:32:39.254 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-23T23:32:39.254 INFO:teuthology.orchestra.run.smithi099.stdout:Adding key to root@localhost authorized_keys... 2024-03-23T23:32:39.254 INFO:teuthology.orchestra.run.smithi099.stdout:Adding host smithi099... 2024-03-23T23:32:39.566 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:39 smithi099 ceph-mon[31715]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:40.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:40 smithi099 ceph-mon[31715]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:42.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:42 smithi099 ceph-mon[31715]: Deploying cephadm binary to smithi099 2024-03-23T23:32:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Added host 'smithi099' with addr '172.21.15.99' 2024-03-23T23:32:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.882+0000 7f9b7dc00700 1 Processor -- start 2024-03-23T23:32:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.882+0000 7f9b7dc00700 1 -- start start 2024-03-23T23:32:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.882+0000 7f9b7dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.882+0000 7f9b7dc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b78082410 con 0x7f9b78081a20 2024-03-23T23:32:47.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.883+0000 7f9b76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:47.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.883+0000 7f9b76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55484/0 (socket says 172.21.15.99:55484) 2024-03-23T23:32:47.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.883+0000 7f9b76a00700 1 -- 172.21.15.99:0/1700724467 learned_addr learned my addr 172.21.15.99:0/1700724467 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:47.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.883+0000 7f9b76a00700 1 -- 172.21.15.99:0/1700724467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7810f290 con 0x7f9b78081a20 2024-03-23T23:32:47.252 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.883+0000 7f9b76a00700 1 --2- 172.21.15.99:0/1700724467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098a0 tx=0x7f9b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df6adea844efe53 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b76000700 1 -- 172.21.15.99:0/1700724467 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68004160 con 0x7f9b78081a20 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b76000700 1 -- 172.21.15.99:0/1700724467 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b68004300 con 0x7f9b78081a20 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b76000700 1 -- 172.21.15.99:0/1700724467 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68011b50 con 0x7f9b78081a20 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 -- 172.21.15.99:0/1700724467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 msgr2=0x7f9b78081e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/1700724467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098a0 tx=0x7f9b68009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 -- 172.21.15.99:0/1700724467 shutdown_connections 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/1700724467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b78081e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 -- 172.21.15.99:0/1700724467 >> 172.21.15.99:0/1700724467 conn(0x7f9b7807d0d0 msgr2=0x7f9b7807f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.884+0000 7f9b7dc00700 1 -- 172.21.15.99:0/1700724467 shutdown_connections 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.885+0000 7f9b7dc00700 1 -- 172.21.15.99:0/1700724467 wait complete. 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b7dc00700 1 Processor -- start 2024-03-23T23:32:47.255 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b7dc00700 1 -- start start 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b7dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b7dc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b78112640 con 0x7f9b78081a20 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55492/0 (socket says 172.21.15.99:55492) 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.886+0000 7f9b76a00700 1 -- 172.21.15.99:0/159954347 learned_addr learned my addr 172.21.15.99:0/159954347 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b76a00700 1 -- 172.21.15.99:0/159954347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b781a2560 con 0x7f9b78081a20 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b76a00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b68011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68003b20 con 0x7f9b78081a20 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b68003cc0 con 0x7f9b78081a20 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b781a27f0 con 0x7f9b78081a20 2024-03-23T23:32:47.256 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68024910 con 0x7f9b78081a20 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.887+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b781a54e0 con 0x7f9b78081a20 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.888+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9b68022020 con 0x7f9b78081a20 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.888+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b780637c0 con 0x7f9b78081a20 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.889+0000 7f9b6f400700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 0x7f9b5c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.889+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9b68011e00 con 0x7f9b78081a20 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.889+0000 7f9b6fe00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 0x7f9b5c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.890+0000 7f9b6fe00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 0x7f9b5c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b60006fd0 tx=0x7f9b60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:47.258 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:39.894+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9b6801acd0 con 0x7f9b78081a20 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:40.123+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b7807e340 con 0x7f9b5c038f70 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.441+0000 7f9b6f400700 1 -- 172.21.15.99:0/159954347 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9b7807e340 con 0x7f9b5c038f70 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 msgr2=0x7f9b5c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 0x7f9b5c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b60006fd0 tx=0x7f9b60006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 msgr2=0x7f9b781a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b68011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 shutdown_connections 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f9b5c038f70 0x7f9b5c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 --2- 172.21.15.99:0/159954347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b78081a20 0x7f9b781a2020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.444+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 >> 172.21.15.99:0/159954347 conn(0x7f9b7807d0d0 msgr2=0x7f9b7807dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.445+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 shutdown_connections 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:46.445+0000 7f9b7dc00700 1 -- 172.21.15.99:0/159954347 wait complete. 2024-03-23T23:32:47.259 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mon service... 2024-03-23T23:32:47.600 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:47 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:47.600 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:47 smithi099 ceph-mon[31715]: Added host smithi099 2024-03-23T23:32:47.600 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:47 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.932+0000 7f99e8e00700 1 Processor -- start 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.932+0000 7f99e8e00700 1 -- start start 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.933+0000 7f99e8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.933+0000 7f99e8e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e4108470 con 0x7f99e4107a80 2024-03-23T23:32:49.023 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.933+0000 7f99e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.933+0000 7f99e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55506/0 (socket says 172.21.15.99:55506) 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.933+0000 7f99e1e00700 1 -- 172.21.15.99:0/1471429524 learned_addr learned my addr 172.21.15.99:0/1471429524 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.934+0000 7f99e1e00700 1 -- 172.21.15.99:0/1471429524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4108c80 con 0x7f99e4107a80 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.934+0000 7f99e1e00700 1 --2- 172.21.15.99:0/1471429524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99d40098a0 tx=0x7f99d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbc36cc287445d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.934+0000 7f99e0a00700 1 -- 172.21.15.99:0/1471429524 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99d4004160 con 0x7f99e4107a80 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.934+0000 7f99e0a00700 1 -- 172.21.15.99:0/1471429524 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d4004300 con 0x7f99e4107a80 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.934+0000 7f99e0a00700 1 -- 172.21.15.99:0/1471429524 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99d4011b50 con 0x7f99e4107a80 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 -- 172.21.15.99:0/1471429524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 msgr2=0x7f99e4107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 --2- 172.21.15.99:0/1471429524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99d40098a0 tx=0x7f99d4009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:49.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 -- 172.21.15.99:0/1471429524 shutdown_connections 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 --2- 172.21.15.99:0/1471429524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e4107ea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 -- 172.21.15.99:0/1471429524 >> 172.21.15.99:0/1471429524 conn(0x7f99e4103000 msgr2=0x7f99e4105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 -- 172.21.15.99:0/1471429524 shutdown_connections 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.935+0000 7f99e8e00700 1 -- 172.21.15.99:0/1471429524 wait complete. 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.936+0000 7f99e8e00700 1 Processor -- start 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.936+0000 7f99e8e00700 1 -- start start 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e8e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e410c030 con 0x7f99e4107a80 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55522/0 (socket says 172.21.15.99:55522) 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e1e00700 1 -- 172.21.15.99:0/70657711 learned_addr learned my addr 172.21.15.99:0/70657711 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e1e00700 1 -- 172.21.15.99:0/70657711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e419e1e0 con 0x7f99e4107a80 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.937+0000 7f99e1e00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000c00 tx=0x7f99d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.938+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99d4003b20 con 0x7f99e4107a80 2024-03-23T23:32:49.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.938+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d4003cc0 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.938+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99d4024910 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.938+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e419e470 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.938+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e41a1160 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.939+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f99d4022020 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.939+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e40637c0 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.940+0000 7f99daa00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 0x7f99c803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.940+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99d4011e00 con 0x7f99e4107a80 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.940+0000 7f99e1400700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 0x7f99c803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.941+0000 7f99e1400700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 0x7f99c803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006fd0 tx=0x7f99d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:49.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:47.945+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99d404c900 con 0x7f99e4107a80 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.189+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f99e4104260 con 0x7f99c8038f70 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.197+0000 7f99daa00700 1 -- 172.21.15.99:0/70657711 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f99e4104260 con 0x7f99c8038f70 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.200+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 msgr2=0x7f99c803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.200+0000 7f99e8e00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 0x7f99c803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006fd0 tx=0x7f99d0006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.200+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 msgr2=0x7f99e419dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.200+0000 7f99e8e00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99d4000c00 tx=0x7f99d4011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.200+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 shutdown_connections 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.201+0000 7f99e8e00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f99c8038f70 0x7f99c803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.201+0000 7f99e8e00700 1 --2- 172.21.15.99:0/70657711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4107a80 0x7f99e419dca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:49.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.201+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 >> 172.21.15.99:0/70657711 conn(0x7f99e4103000 msgr2=0x7f99e4103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:49.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.201+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 shutdown_connections 2024-03-23T23:32:49.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:48.201+0000 7f99e8e00700 1 -- 172.21.15.99:0/70657711 wait complete. 2024-03-23T23:32:49.032 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mgr service... 2024-03-23T23:32:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:49 smithi099 ceph-mon[31715]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:49 smithi099 ceph-mon[31715]: Saving service mon spec with placement count:5 2024-03-23T23:32:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:49 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:51.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:51 smithi099 ceph-mon[31715]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:32:51.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:51 smithi099 ceph-mon[31715]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:32:51.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:51 smithi099 ceph-mon[31715]: Saving service mgr spec with placement count:2 2024-03-23T23:32:51.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:51 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:51.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-23T23:32:51.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.774+0000 7f0924800700 1 Processor -- start 2024-03-23T23:32:51.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.774+0000 7f0924800700 1 -- start start 2024-03-23T23:32:51.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0924800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:51.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0924800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c108470 con 0x7f091c107a80 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0921e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0921e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52310/0 (socket says 172.21.15.99:52310) 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0921e00700 1 -- 172.21.15.99:0/1993371939 learned_addr learned my addr 172.21.15.99:0/1993371939 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.775+0000 7f0921e00700 1 -- 172.21.15.99:0/1993371939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c108c80 con 0x7f091c107a80 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0921e00700 1 --2- 172.21.15.99:0/1993371939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09100098a0 tx=0x7f0910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1c839d0d99187f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0920a00700 1 -- 172.21.15.99:0/1993371939 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910004160 con 0x7f091c107a80 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0920a00700 1 -- 172.21.15.99:0/1993371939 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0910004300 con 0x7f091c107a80 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0920a00700 1 -- 172.21.15.99:0/1993371939 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910011b50 con 0x7f091c107a80 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0924800700 1 -- 172.21.15.99:0/1993371939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 msgr2=0x7f091c107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.776+0000 7f0924800700 1 --2- 172.21.15.99:0/1993371939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09100098a0 tx=0x7f0910009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.777+0000 7f0924800700 1 -- 172.21.15.99:0/1993371939 shutdown_connections 2024-03-23T23:32:51.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.777+0000 7f0924800700 1 --2- 172.21.15.99:0/1993371939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c107ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.777+0000 7f0924800700 1 -- 172.21.15.99:0/1993371939 >> 172.21.15.99:0/1993371939 conn(0x7f091c103020 msgr2=0x7f091c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.777+0000 7f0924800700 1 -- 172.21.15.99:0/1993371939 shutdown_connections 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.777+0000 7f0924800700 1 -- 172.21.15.99:0/1993371939 wait complete. 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0924800700 1 Processor -- start 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0924800700 1 -- start start 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0924800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0924800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c10c030 con 0x7f091c107a80 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0921e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0921e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52312/0 (socket says 172.21.15.99:52312) 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.778+0000 7f0921e00700 1 -- 172.21.15.99:0/1908153656 learned_addr learned my addr 172.21.15.99:0/1908153656 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0921e00700 1 -- 172.21.15.99:0/1908153656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c07e0d0 con 0x7f091c107a80 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0921e00700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0910000c00 tx=0x7f0910011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910003b20 con 0x7f091c107a80 2024-03-23T23:32:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0910003cc0 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910024910 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c07c2c0 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.779+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c07c800 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.780+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0910022020 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.780+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c0637c0 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.781+0000 7f0916a00700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 0x7f090403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.781+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0910011e00 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.781+0000 7f0921400700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 0x7f090403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.782+0000 7f0921400700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 0x7f090403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f090c006fd0 tx=0x7f090c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:49.786+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0910016260 con 0x7f091c107a80 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.025+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f091c1042c0 con 0x7f0904038f70 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.036+0000 7f0916a00700 1 -- 172.21.15.99:0/1908153656 <== mgr.14120 v2:172.21.15.99:6800/2749486918 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f091c1042c0 con 0x7f0904038f70 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 msgr2=0x7f090403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:51.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 0x7f090403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f090c006fd0 tx=0x7f090c006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 msgr2=0x7f091c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0910000c00 tx=0x7f0910011c10 comp rx=0 tx=0).stop 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 shutdown_connections 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0904038f70 0x7f090403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 --2- 172.21.15.99:0/1908153656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c107a80 0x7f091c07db90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 >> 172.21.15.99:0/1908153656 conn(0x7f091c103020 msgr2=0x7f091c103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 shutdown_connections 2024-03-23T23:32:51.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:50.039+0000 7f0924800700 1 -- 172.21.15.99:0/1908153656 wait complete. 2024-03-23T23:32:52.534 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:52 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:52.535 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:52 smithi099 ceph-mon[31715]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:32:52.535 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:52 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:53.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.536+0000 7f4b65200700 1 Processor -- start 2024-03-23T23:32:53.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.536+0000 7f4b65200700 1 -- start start 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.536+0000 7f4b65200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.536+0000 7f4b65200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6007c8d0 con 0x7f4b6007dea0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.537+0000 7f4b5de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.537+0000 7f4b5de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52320/0 (socket says 172.21.15.99:52320) 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.537+0000 7f4b5de00700 1 -- 172.21.15.99:0/998155391 learned_addr learned my addr 172.21.15.99:0/998155391 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.537+0000 7f4b5de00700 1 -- 172.21.15.99:0/998155391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6007ca50 con 0x7f4b6007dea0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.538+0000 7f4b5de00700 1 --2- 172.21.15.99:0/998155391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f886a016badcaee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.538+0000 7f4b5ca00700 1 -- 172.21.15.99:0/998155391 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50004160 con 0x7f4b6007dea0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.538+0000 7f4b5ca00700 1 -- 172.21.15.99:0/998155391 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b50004300 con 0x7f4b6007dea0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.538+0000 7f4b5ca00700 1 -- 172.21.15.99:0/998155391 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50011b20 con 0x7f4b6007dea0 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 -- 172.21.15.99:0/998155391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 msgr2=0x7f4b6007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 --2- 172.21.15.99:0/998155391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 -- 172.21.15.99:0/998155391 shutdown_connections 2024-03-23T23:32:53.695 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 --2- 172.21.15.99:0/998155391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6007c300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:53.696 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 -- 172.21.15.99:0/998155391 >> 172.21.15.99:0/998155391 conn(0x7f4b60103020 msgr2=0x7f4b60105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 -- 172.21.15.99:0/998155391 shutdown_connections 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.539+0000 7f4b65200700 1 -- 172.21.15.99:0/998155391 wait complete. 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.540+0000 7f4b65200700 1 Processor -- start 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.540+0000 7f4b65200700 1 -- start start 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.540+0000 7f4b65200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.540+0000 7f4b65200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6010e100 con 0x7f4b6007dea0 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.541+0000 7f4b5de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:53.698 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.541+0000 7f4b5de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52332/0 (socket says 172.21.15.99:52332) 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.541+0000 7f4b5de00700 1 -- 172.21.15.99:0/3329562602 learned_addr learned my addr 172.21.15.99:0/3329562602 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.541+0000 7f4b5de00700 1 -- 172.21.15.99:0/3329562602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6019e140 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.541+0000 7f4b5de00700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b500116a0 tx=0x7f4b50011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.542+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b50003a40 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.542+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b50003be0 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.542+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6019e3d0 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.542+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b5001a750 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.542+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6019e830 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.543+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f4b5001aa50 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.543+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b60069250 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.544+0000 7f4b56a00700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 0x7f4b4403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.544+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b5004d580 con 0x7f4b6007dea0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.544+0000 7f4b5d400700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 0x7f4b4403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:53.699 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.545+0000 7f4b5d400700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 0x7f4b4403b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006fd0 tx=0x7f4b4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.550+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b50011e20 con 0x7f4b6007dea0 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.771+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f4b601a10c0 con 0x7f4b6007dea0 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.776+0000 7f4b56a00700 1 -- 172.21.15.99:0/3329562602 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4b5001adf0 con 0x7f4b6007dea0 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 msgr2=0x7f4b4403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 0x7f4b4403b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006fd0 tx=0x7f4b4c006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 msgr2=0x7f4b6019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b500116a0 tx=0x7f4b50011ba0 comp rx=0 tx=0).stop 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 shutdown_connections 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f4b44038f20 0x7f4b4403b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 --2- 172.21.15.99:0/3329562602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6007dea0 0x7f4b6019dc00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.780+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 >> 172.21.15.99:0/3329562602 conn(0x7f4b60103020 msgr2=0x7f4b60103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.781+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 shutdown_connections 2024-03-23T23:32:53.700 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:52.781+0000 7f4b65200700 1 -- 172.21.15.99:0/3329562602 wait complete. 2024-03-23T23:32:53.937 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3329562602' entity='client.admin' 2024-03-23T23:32:54.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:54 smithi099 ceph-mon[31715]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:32:54.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:54 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/795962706' entity='client.admin' 2024-03-23T23:32:56.000 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.347+0000 7f3ab2200700 1 Processor -- start 2024-03-23T23:32:56.000 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.347+0000 7f3ab2200700 1 -- start start 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.347+0000 7f3ab2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.347+0000 7f3ab2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac108470 con 0x7f3aac107a80 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.347+0000 7f3aab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.348+0000 7f3aab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52336/0 (socket says 172.21.15.99:52336) 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.348+0000 7f3aab400700 1 -- 172.21.15.99:0/2477193005 learned_addr learned my addr 172.21.15.99:0/2477193005 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.348+0000 7f3aab400700 1 -- 172.21.15.99:0/2477193005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac108c80 con 0x7f3aac107a80 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.348+0000 7f3aab400700 1 --2- 172.21.15.99:0/2477193005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e80 tx=0x7f3a9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4373f08dd64593 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.349+0000 7f3aaa000700 1 -- 172.21.15.99:0/2477193005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a9c017070 con 0x7f3aac107a80 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.349+0000 7f3aaa000700 1 -- 172.21.15.99:0/2477193005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c00b8c0 con 0x7f3aac107a80 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.349+0000 7f3aaa000700 1 -- 172.21.15.99:0/2477193005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a9c0045c0 con 0x7f3aac107a80 2024-03-23T23:32:56.002 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 -- 172.21.15.99:0/2477193005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 msgr2=0x7f3aac107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:56.003 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 --2- 172.21.15.99:0/2477193005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e80 tx=0x7f3a9c00b330 comp rx=0 tx=0).stop 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 -- 172.21.15.99:0/2477193005 shutdown_connections 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 --2- 172.21.15.99:0/2477193005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac107ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 -- 172.21.15.99:0/2477193005 >> 172.21.15.99:0/2477193005 conn(0x7f3aac1030c0 msgr2=0x7f3aac1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 -- 172.21.15.99:0/2477193005 shutdown_connections 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.350+0000 7f3ab2200700 1 -- 172.21.15.99:0/2477193005 wait complete. 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.351+0000 7f3ab2200700 1 Processor -- start 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3ab2200700 1 -- start start 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3ab2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3ab2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac10c030 con 0x7f3aac107a80 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3aab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:56.008 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3aab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52348/0 (socket says 172.21.15.99:52348) 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.352+0000 7f3aab400700 1 -- 172.21.15.99:0/795962706 learned_addr learned my addr 172.21.15.99:0/795962706 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3aab400700 1 -- 172.21.15.99:0/795962706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac07e0d0 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3aab400700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00bca0 tx=0x7f3a9c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a9c020020 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c003d90 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac07c2c0 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a9c017040 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.353+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac07c800 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.354+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f3a9c023020 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.355+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c005360 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.355+0000 7f3aa3e00700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 0x7f3a9003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.355+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a9c00f050 con 0x7f3aac107a80 2024-03-23T23:32:56.009 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.355+0000 7f3aaaa00700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 0x7f3a9003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.356+0000 7f3aaaa00700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 0x7f3a9003b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.363+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a9c02a020 con 0x7f3aac107a80 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.587+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f3a8c005650 con 0x7f3aac107a80 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.592+0000 7f3aa3e00700 1 -- 172.21.15.99:0/795962706 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f3a9c025400 con 0x7f3aac107a80 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 msgr2=0x7f3a9003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 0x7f3a9003b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 msgr2=0x7f3aac07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00bca0 tx=0x7f3a9c00bd80 comp rx=0 tx=0).stop 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 shutdown_connections 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f3a90038f20 0x7f3a9003b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 --2- 172.21.15.99:0/795962706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3aac107a80 0x7f3aac07db90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 >> 172.21.15.99:0/795962706 conn(0x7f3aac1030c0 msgr2=0x7f3aac103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:56.010 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 shutdown_connections 2024-03-23T23:32:56.011 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:54.595+0000 7f3ab2200700 1 -- 172.21.15.99:0/795962706 wait complete. 2024-03-23T23:32:56.011 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling the dashboard module... 2024-03-23T23:32:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:56 smithi099 ceph-mon[31715]: from='mgr.14120 172.21.15.99:0/3796201788' entity='mgr.x' 2024-03-23T23:32:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:56 smithi099 ceph-mon[31715]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:32:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:57 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2195136808' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-23T23:32:58.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: ignoring --setuser ceph since I am not root 2024-03-23T23:32:58.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: ignoring --setgroup ceph since I am not root 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.648+0000 7f46d6a00700 1 Processor -- start 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.648+0000 7f46d6a00700 1 -- start start 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.649+0000 7f46d6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.649+0000 7f46d6a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d007c8d0 con 0x7f46d007dea0 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.649+0000 7f46cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.649+0000 7f46cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52356/0 (socket says 172.21.15.99:52356) 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.649+0000 7f46cfe00700 1 -- 172.21.15.99:0/1562342760 learned_addr learned my addr 172.21.15.99:0/1562342760 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.650+0000 7f46cfe00700 1 -- 172.21.15.99:0/1562342760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d007ca50 con 0x7f46d007dea0 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.650+0000 7f46cfe00700 1 --2- 172.21.15.99:0/1562342760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098a0 tx=0x7f46c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf71db389a1a5b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:58.652 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.650+0000 7f46cea00700 1 -- 172.21.15.99:0/1562342760 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46c0004160 con 0x7f46d007dea0 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.650+0000 7f46cea00700 1 -- 172.21.15.99:0/1562342760 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46c0004300 con 0x7f46d007dea0 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.650+0000 7f46cea00700 1 -- 172.21.15.99:0/1562342760 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46c0011b20 con 0x7f46d007dea0 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.651+0000 7f46d6a00700 1 -- 172.21.15.99:0/1562342760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 msgr2=0x7f46d007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.651+0000 7f46d6a00700 1 --2- 172.21.15.99:0/1562342760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46c00098a0 tx=0x7f46c0009bb0 comp rx=0 tx=0).stop 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.651+0000 7f46d6a00700 1 -- 172.21.15.99:0/1562342760 shutdown_connections 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.651+0000 7f46d6a00700 1 --2- 172.21.15.99:0/1562342760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d007c300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.651+0000 7f46d6a00700 1 -- 172.21.15.99:0/1562342760 >> 172.21.15.99:0/1562342760 conn(0x7f46d0103000 msgr2=0x7f46d0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.652+0000 7f46d6a00700 1 -- 172.21.15.99:0/1562342760 shutdown_connections 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.652+0000 7f46d6a00700 1 -- 172.21.15.99:0/1562342760 wait complete. 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.653+0000 7f46d6a00700 1 Processor -- start 2024-03-23T23:32:58.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.653+0000 7f46d6a00700 1 -- start start 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.653+0000 7f46d6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.653+0000 7f46d6a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d010e100 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.653+0000 7f46cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.654+0000 7f46cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52360/0 (socket says 172.21.15.99:52360) 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.654+0000 7f46cfe00700 1 -- 172.21.15.99:0/2195136808 learned_addr learned my addr 172.21.15.99:0/2195136808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.654+0000 7f46cfe00700 1 -- 172.21.15.99:0/2195136808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d019e0f0 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.654+0000 7f46cfe00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46c00117e0 tx=0x7f46c0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.654+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46c0003ba0 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.655+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46c001a910 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.655+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46c00248f0 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.655+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d019e380 con 0x7f46d007dea0 2024-03-23T23:32:58.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.655+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d019e7e0 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.656+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f46c0022020 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.656+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d00637c0 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.656+0000 7f46ccc00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 0x7f46b403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.656+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f46c004d340 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.657+0000 7f46cf400700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 0x7f46b403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.657+0000 7f46cf400700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 0x7f46b403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.663+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f46c001ab70 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:56.941+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f46d01a11b0 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.792+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f46c004cc90 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.795+0000 7f46ccc00700 1 -- 172.21.15.99:0/2195136808 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f46c002d3f0 con 0x7f46d007dea0 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 msgr2=0x7f46b403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 0x7f46b403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).stop 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 msgr2=0x7f46d019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:32:58.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46c00117e0 tx=0x7f46c0011ba0 comp rx=0 tx=0).stop 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 shutdown_connections 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f46b4038f70 0x7f46b403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 --2- 172.21.15.99:0/2195136808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d007dea0 0x7f46d019dbb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.797+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 >> 172.21.15.99:0/2195136808 conn(0x7f46d0103000 msgr2=0x7f46d0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.798+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 shutdown_connections 2024-03-23T23:32:58.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:57.798+0000 7f46d6a00700 1 -- 172.21.15.99:0/2195136808 wait complete. 2024-03-23T23:32:59.025 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:58.611+0000 7f706b875200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T23:32:59.025 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:58.747+0000 7f706b875200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T23:32:59.025 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:58 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2195136808' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-23T23:32:59.025 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:58 smithi099 ceph-mon[31715]: mgrmap e9: x(active, since 27s) 2024-03-23T23:32:59.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:32:59 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/267072297' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T23:32:59.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:59.549+0000 7f706b875200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T23:32:59.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:59.677+0000 7f706b875200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T23:33:00.116 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:59.862+0000 7f706b875200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T23:33:00.117 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:32:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:32:59.987+0000 7f706b875200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T23:33:00.117 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:00.095+0000 7f706b875200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T23:33:00.451 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:33:00.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-23T23:33:00.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.346+0000 7f1ddda00700 1 Processor -- start 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.346+0000 7f1ddda00700 1 -- start start 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.346+0000 7f1ddda00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.346+0000 7f1ddda00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8108480 con 0x7f1dd8107a90 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.347+0000 7f1dd6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.347+0000 7f1dd6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37770/0 (socket says 172.21.15.99:37770) 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.347+0000 7f1dd6a00700 1 -- 172.21.15.99:0/3874260519 learned_addr learned my addr 172.21.15.99:0/3874260519 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.347+0000 7f1dd6a00700 1 -- 172.21.15.99:0/3874260519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8108c90 con 0x7f1dd8107a90 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.347+0000 7f1dd6a00700 1 --2- 172.21.15.99:0/3874260519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80098a0 tx=0x7f1dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1653151af7d3924 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.348+0000 7f1dd5600700 1 -- 172.21.15.99:0/3874260519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc8004160 con 0x7f1dd8107a90 2024-03-23T23:33:00.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.348+0000 7f1dd5600700 1 -- 172.21.15.99:0/3874260519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc8004300 con 0x7f1dd8107a90 2024-03-23T23:33:00.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.348+0000 7f1dd5600700 1 -- 172.21.15.99:0/3874260519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc8011b20 con 0x7f1dd8107a90 2024-03-23T23:33:00.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 -- 172.21.15.99:0/3874260519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 msgr2=0x7f1dd8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:00.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 --2- 172.21.15.99:0/3874260519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80098a0 tx=0x7f1dc8009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 -- 172.21.15.99:0/3874260519 shutdown_connections 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 --2- 172.21.15.99:0/3874260519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd8107eb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 -- 172.21.15.99:0/3874260519 >> 172.21.15.99:0/3874260519 conn(0x7f1dd8103050 msgr2=0x7f1dd8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 -- 172.21.15.99:0/3874260519 shutdown_connections 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.349+0000 7f1ddda00700 1 -- 172.21.15.99:0/3874260519 wait complete. 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.350+0000 7f1ddda00700 1 Processor -- start 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.350+0000 7f1ddda00700 1 -- start start 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.350+0000 7f1ddda00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.350+0000 7f1ddda00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd810c040 con 0x7f1dd8107a90 2024-03-23T23:33:00.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dd6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dd6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37786/0 (socket says 172.21.15.99:37786) 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dd6a00700 1 -- 172.21.15.99:0/267072297 learned_addr learned my addr 172.21.15.99:0/267072297 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dd6a00700 1 -- 172.21.15.99:0/267072297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd807e0d0 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dd6a00700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80116a0 tx=0x7f1dc8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc8003a40 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.351+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc8003be0 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.352+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd807c2c0 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.352+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc801a720 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.352+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd807c780 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.353+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8069250 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.353+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f1dc8024aa0 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.354+0000 7f1dcf400700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 0x7f1dbc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.354+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1dc804d560 con 0x7f1dd8107a90 2024-03-23T23:33:00.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.354+0000 7f1dd6000700 1 -- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 msgr2=0x7f1dbc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.354+0000 7f1dd6000700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 0x7f1dbc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.359+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1dc8011e20 con 0x7f1dd8107a90 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.555+0000 7f1dd6000700 1 -- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 msgr2=0x7f1dbc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.555+0000 7f1dd6000700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 0x7f1dbc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.636+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1dd807d020 con 0x7f1dd8107a90 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.636+0000 7f1dcf400700 1 -- 172.21.15.99:0/267072297 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1dc804ce30 con 0x7f1dd8107a90 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 msgr2=0x7f1dbc03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 0x7f1dbc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 msgr2=0x7f1dd807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80116a0 tx=0x7f1dc8011ba0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 shutdown_connections 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f1dbc038f20 0x7f1dbc03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 --2- 172.21.15.99:0/267072297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dd8107a90 0x7f1dd807db90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 >> 172.21.15.99:0/267072297 conn(0x7f1dd8103050 msgr2=0x7f1dd8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:00.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 shutdown_connections 2024-03-23T23:33:00.462 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:32:59.640+0000 7f1ddda00700 1 -- 172.21.15.99:0/267072297 wait complete. 2024-03-23T23:33:00.462 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-03-23T23:33:00.462 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 9... 2024-03-23T23:33:00.696 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:00.438+0000 7f706b875200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T23:33:00.696 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:00.583+0000 7f706b875200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T23:33:00.697 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:00.696+0000 7f706b875200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T23:33:01.045 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:00.826+0000 7f706b875200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T23:33:01.045 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:01.046+0000 7f706b875200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T23:33:02.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:02.298+0000 7f706b875200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T23:33:02.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:02.466+0000 7f706b875200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T23:33:03.242 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:02.877+0000 7f706b875200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T23:33:03.242 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:02.993+0000 7f706b875200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T23:33:03.242 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:03.241+0000 7f706b875200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:33:03.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:03.484+0000 7f706b875200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T23:33:04.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:03.854+0000 7f706b875200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:33:04.469 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:04 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:04.169+0000 7f706b875200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T23:33:04.469 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:04 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:04.468+0000 7f706b875200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T23:33:05.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:05 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:05.473+0000 7f706b875200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T23:33:06.581 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:06.165+0000 7f706b875200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T23:33:06.582 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:06.302+0000 7f706b875200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:33:06.580+0000 7f706b875200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: Active manager daemon x restarted 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: Activating manager daemon x 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: osdmap e3: 0 total, 0 up, 0 in 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: mgrmap e10: x(active, starting, since 0.0048329s) 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T23:33:06.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T23:33:06.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T23:33:06.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:06 smithi099 ceph-mon[31715]: Manager daemon x is now available 2024-03-23T23:33:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:07.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T23:33:07.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T23:33:07.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:07 smithi099 ceph-mon[31715]: mgrmap e11: x(active, since 1.01316s) 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.080+0000 7f0bd4200700 1 Processor -- start 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.081+0000 7f0bd4200700 1 -- start start 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.081+0000 7f0bd4200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.081+0000 7f0bd4200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc07c8f0 con 0x7f0bcc07dec0 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.082+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:08.526 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.082+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37792/0 (socket says 172.21.15.99:37792) 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.082+0000 7f0bd1800700 1 -- 172.21.15.99:0/3158607778 learned_addr learned my addr 172.21.15.99:0/3158607778 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.082+0000 7f0bd1800700 1 -- 172.21.15.99:0/3158607778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc07ca70 con 0x7f0bcc07dec0 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.083+0000 7f0bd1800700 1 --2- 172.21.15.99:0/3158607778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de19624130b7af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.083+0000 7f0bcbe00700 1 -- 172.21.15.99:0/3158607778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0004160 con 0x7f0bcc07dec0 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.083+0000 7f0bcbe00700 1 -- 172.21.15.99:0/3158607778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc0004300 con 0x7f0bcc07dec0 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.083+0000 7f0bcbe00700 1 -- 172.21.15.99:0/3158607778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0011b20 con 0x7f0bcc07dec0 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 -- 172.21.15.99:0/3158607778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 msgr2=0x7f0bcc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 --2- 172.21.15.99:0/3158607778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098a0 tx=0x7f0bc0009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 -- 172.21.15.99:0/3158607778 shutdown_connections 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 --2- 172.21.15.99:0/3158607778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc07c320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 -- 172.21.15.99:0/3158607778 >> 172.21.15.99:0/3158607778 conn(0x7f0bcc103020 msgr2=0x7f0bcc105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 -- 172.21.15.99:0/3158607778 shutdown_connections 2024-03-23T23:33:08.527 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.084+0000 7f0bd4200700 1 -- 172.21.15.99:0/3158607778 wait complete. 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.085+0000 7f0bd4200700 1 Processor -- start 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.085+0000 7f0bd4200700 1 -- start start 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.085+0000 7f0bd4200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.085+0000 7f0bd4200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc10e0e0 con 0x7f0bcc07dec0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37806/0 (socket says 172.21.15.99:37806) 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd1800700 1 -- 172.21.15.99:0/2109755692 learned_addr learned my addr 172.21.15.99:0/2109755692 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd1800700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc19e190 con 0x7f0bcc07dec0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd1800700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c00 tx=0x7f0bc0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0003b60 con 0x7f0bcc07dec0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc001a910 con 0x7f0bcc07dec0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.086+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc19e420 con 0x7f0bcc07dec0 2024-03-23T23:33:08.528 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.087+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0024930 con 0x7f0bcc07dec0 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.087+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc19e880 con 0x7f0bcc07dec0 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f0bc0022020 con 0x7f0bcc07dec0 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bca000700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bb403bba0 con 0x7f0bb4038fd0 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0bc004d0c0 con 0x7f0bcc07dec0 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bd0e00700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.088+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.289+0000 7f0bd0e00700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.289+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.690+0000 7f0bd0e00700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:01.690+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:02.491+0000 7f0bd0e00700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:02.491+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T23:33:08.529 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:04.091+0000 7f0bd0e00700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/2749486918 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:04.091+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:06.589+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f0bc002d3f0 con 0x7f0bcc07dec0 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:06.589+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 msgr2=0x7f0bb403b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:06.589+0000 7f0bca000700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.594+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f0bc004ec70 con 0x7f0bcc07dec0 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.594+0000 7f0bca000700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 0x7f0bb403ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.594+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bc000af30 con 0x7f0bb403ca80 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.595+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 0x7f0bb403ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.596+0000 7f0bd0e00700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 0x7f0bb403ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8003a10 tx=0x7f0bb8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.596+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f0bc000af30 con 0x7f0bb403ca80 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.604+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0bcc069250 con 0x7f0bb403ca80 2024-03-23T23:33:08.530 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bca000700 1 -- 172.21.15.99:0/2109755692 <== mgr.14152 v2:172.21.15.99:6800/3957482815 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0bcc069250 con 0x7f0bb403ca80 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 msgr2=0x7f0bb403ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 0x7f0bb403ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8003a10 tx=0x7f0bb8009270 comp rx=0 tx=0).stop 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 msgr2=0x7f0bcc19dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c00 tx=0x7f0bc0011c10 comp rx=0 tx=0).stop 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 shutdown_connections 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb403ca80 0x7f0bb403ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bcc07dec0 0x7f0bcc19dc50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 --2- 172.21.15.99:0/2109755692 >> [v2:172.21.15.99:6800/2749486918,v1:172.21.15.99:6801/2749486918] conn(0x7f0bb4038fd0 0x7f0bb403b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 >> 172.21.15.99:0/2109755692 conn(0x7f0bcc103020 msgr2=0x7f0bcc103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 shutdown_connections 2024-03-23T23:33:08.531 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:07.605+0000 7f0bd4200700 1 -- 172.21.15.99:0/2109755692 wait complete. 2024-03-23T23:33:08.532 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 9 is available 2024-03-23T23:33:08.532 INFO:teuthology.orchestra.run.smithi099.stdout:Generating a dashboard self-signed certificate... 2024-03-23T23:33:08.935 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: [23/Mar/2024:23:33:07] ENGINE Bus STARTING 2024-03-23T23:33:08.935 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: [23/Mar/2024:23:33:07] ENGINE Serving on http://172.21.15.99:8765 2024-03-23T23:33:08.935 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: [23/Mar/2024:23:33:07] ENGINE Serving on https://172.21.15.99:7150 2024-03-23T23:33:08.935 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: [23/Mar/2024:23:33:07] ENGINE Bus STARTED 2024-03-23T23:33:08.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T23:33:08.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:08 smithi099 ceph-mon[31715]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T23:33:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:09 smithi099 ceph-mon[31715]: mgrmap e12: x(active, since 2s) 2024-03-23T23:33:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:10.772 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-23T23:33:10.772 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.231+0000 7f5aca600700 1 Processor -- start 2024-03-23T23:33:10.772 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.231+0000 7f5aca600700 1 -- start start 2024-03-23T23:33:10.772 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5aca600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5aca600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4082210 con 0x7f5ac40818b0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5ac9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5ac9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60476/0 (socket says 172.21.15.99:60476) 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5ac9200700 1 -- 172.21.15.99:0/3882200569 learned_addr learned my addr 172.21.15.99:0/3882200569 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.232+0000 7f5ac9200700 1 -- 172.21.15.99:0/3882200569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac410f080 con 0x7f5ac40818b0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.233+0000 7f5ac9200700 1 --2- 172.21.15.99:0/3882200569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80098a0 tx=0x7f5ab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8ee07fc82ed562 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.233+0000 7f5ac3400700 1 -- 172.21.15.99:0/3882200569 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ab8004160 con 0x7f5ac40818b0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.233+0000 7f5ac3400700 1 -- 172.21.15.99:0/3882200569 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab8004300 con 0x7f5ac40818b0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.233+0000 7f5ac3400700 1 -- 172.21.15.99:0/3882200569 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ab8011b20 con 0x7f5ac40818b0 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 -- 172.21.15.99:0/3882200569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 msgr2=0x7f5ac4081cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 --2- 172.21.15.99:0/3882200569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80098a0 tx=0x7f5ab8009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 -- 172.21.15.99:0/3882200569 shutdown_connections 2024-03-23T23:33:10.773 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 --2- 172.21.15.99:0/3882200569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac4081cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 -- 172.21.15.99:0/3882200569 >> 172.21.15.99:0/3882200569 conn(0x7f5ac407ce80 msgr2=0x7f5ac407f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 -- 172.21.15.99:0/3882200569 shutdown_connections 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.234+0000 7f5aca600700 1 -- 172.21.15.99:0/3882200569 wait complete. 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.235+0000 7f5aca600700 1 Processor -- start 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.235+0000 7f5aca600700 1 -- start start 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5aca600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5aca600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4112340 con 0x7f5ac40818b0 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5ac9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5ac9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60488/0 (socket says 172.21.15.99:60488) 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5ac9200700 1 -- 172.21.15.99:0/3693980580 learned_addr learned my addr 172.21.15.99:0/3693980580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5ac9200700 1 -- 172.21.15.99:0/3693980580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac41a2450 con 0x7f5ac40818b0 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.236+0000 7f5ac9200700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8011b60 tx=0x7f5ab8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:10.774 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.237+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ab8003b60 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.237+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab801a910 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.237+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ab8024930 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.237+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac41a26e0 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.237+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac41a2b40 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.238+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f5ab8022020 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.238+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac4069250 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.238+0000 7f5ac1600700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 0x7f5aac03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.238+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ab804d1c0 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.239+0000 7f5ac3e00700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 0x7f5aac03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.239+0000 7f5ac3e00700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 0x7f5aac03b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0006fd0 tx=0x7f5ab0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.244+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ab800b3e0 con 0x7f5ac40818b0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.475+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ac407e0f0 con 0x7f5aac038fd0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.581+0000 7f5ac1600700 1 -- 172.21.15.99:0/3693980580 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5ac407e0f0 con 0x7f5aac038fd0 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 msgr2=0x7f5aac03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:10.775 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 0x7f5aac03b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0006fd0 tx=0x7f5ab0006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 msgr2=0x7f5ac41a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8011b60 tx=0x7f5ab8011c40 comp rx=0 tx=0).stop 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 shutdown_connections 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5aac038fd0 0x7f5aac03b490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 --2- 172.21.15.99:0/3693980580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ac40818b0 0x7f5ac41a1f10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 >> 172.21.15.99:0/3693980580 conn(0x7f5ac407ce80 msgr2=0x7f5ac407d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 shutdown_connections 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:09.585+0000 7f5aca600700 1 -- 172.21.15.99:0/3693980580 wait complete. 2024-03-23T23:33:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial admin user... 2024-03-23T23:33:10.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:10 smithi099 ceph-mon[31715]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:33:10.928 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mQA06cN3zgW1Sh93DdOns.ee8O7naK/YzlICdMplqzTYd60.XQ4AS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711236791, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.439+0000 7f693bc00700 1 Processor -- start 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.439+0000 7f693bc00700 1 -- start start 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.440+0000 7f693bc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.440+0000 7f693bc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693407c910 con 0x7f693407dee0 2024-03-23T23:33:12.595 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.440+0000 7f6939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.440+0000 7f6939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60504/0 (socket says 172.21.15.99:60504) 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.440+0000 7f6939200700 1 -- 172.21.15.99:0/2675737920 learned_addr learned my addr 172.21.15.99:0/2675737920 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.441+0000 7f6939200700 1 -- 172.21.15.99:0/2675737920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693407ca90 con 0x7f693407dee0 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.441+0000 7f6939200700 1 --2- 172.21.15.99:0/2675737920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69280098a0 tx=0x7f6928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a7af21711918d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.441+0000 7f6933400700 1 -- 172.21.15.99:0/2675737920 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6928004160 con 0x7f693407dee0 2024-03-23T23:33:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.441+0000 7f6933400700 1 -- 172.21.15.99:0/2675737920 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6928004300 con 0x7f693407dee0 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.441+0000 7f6933400700 1 -- 172.21.15.99:0/2675737920 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6928011b20 con 0x7f693407dee0 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 -- 172.21.15.99:0/2675737920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 msgr2=0x7f693407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 --2- 172.21.15.99:0/2675737920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69280098a0 tx=0x7f6928009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 -- 172.21.15.99:0/2675737920 shutdown_connections 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 --2- 172.21.15.99:0/2675737920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f693407c340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 -- 172.21.15.99:0/2675737920 >> 172.21.15.99:0/2675737920 conn(0x7f6934103020 msgr2=0x7f6934105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 -- 172.21.15.99:0/2675737920 shutdown_connections 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.442+0000 7f693bc00700 1 -- 172.21.15.99:0/2675737920 wait complete. 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.443+0000 7f693bc00700 1 Processor -- start 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.443+0000 7f693bc00700 1 -- start start 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.443+0000 7f693bc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.443+0000 7f693bc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934109cb0 con 0x7f693407dee0 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.444+0000 7f6939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.444+0000 7f6939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60516/0 (socket says 172.21.15.99:60516) 2024-03-23T23:33:12.601 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.444+0000 7f6939200700 1 -- 172.21.15.99:0/3521566960 learned_addr learned my addr 172.21.15.99:0/3521566960 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.444+0000 7f6939200700 1 -- 172.21.15.99:0/3521566960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69341a2540 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.444+0000 7f6939200700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6928000c00 tx=0x7f6928011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.445+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6928003b60 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.445+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f692801a910 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.445+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69341a27d0 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.445+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6928024930 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.445+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69341a2c30 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.446+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f6928022020 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.446+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934069250 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.446+0000 7f6931600700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 0x7f691c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.447+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f692804d1e0 con 0x7f693407dee0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.447+0000 7f6933e00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 0x7f691c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.447+0000 7f6933e00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 0x7f691c03b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6920006fd0 tx=0x7f6920006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:12.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.451+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f69280212c0 con 0x7f693407dee0 2024-03-23T23:33:12.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.686+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f69341a5640 con 0x7f691c038fd0 2024-03-23T23:33:12.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.923+0000 7f6931600700 1 -- 172.21.15.99:0/3521566960 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f69341a5640 con 0x7f691c038fd0 2024-03-23T23:33:12.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 msgr2=0x7f691c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:12.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 0x7f691c03b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6920006fd0 tx=0x7f6920006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:12.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 msgr2=0x7f69341a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6928000c00 tx=0x7f6928011c10 comp rx=0 tx=0).stop 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 shutdown_connections 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f691c038fd0 0x7f691c03b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 --2- 172.21.15.99:0/3521566960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f693407dee0 0x7f69341a2000 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 >> 172.21.15.99:0/3521566960 conn(0x7f6934103020 msgr2=0x7f6934105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:12.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 shutdown_connections 2024-03-23T23:33:12.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:11.925+0000 7f693bc00700 1 -- 172.21.15.99:0/3521566960 wait complete. 2024-03-23T23:33:12.605 INFO:teuthology.orchestra.run.smithi099.stdout:Fetching dashboard port number... 2024-03-23T23:33:13.182 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:12 smithi099 ceph-mon[31715]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:33:13.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:12 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:13.183 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:12 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:14.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:14 smithi099 ceph-mon[31715]: mgrmap e13: x(active, since 6s) 2024-03-23T23:33:14.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:14 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2631152670' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-23T23:33:14.722 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 8443 2024-03-23T23:33:14.723 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.347+0000 7faa8ac00700 1 Processor -- start 2024-03-23T23:33:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.348+0000 7faa8ac00700 1 -- start start 2024-03-23T23:33:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.348+0000 7faa8ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.348+0000 7faa8ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa84082410 con 0x7faa84081a20 2024-03-23T23:33:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.348+0000 7faa83e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.349+0000 7faa83e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60526/0 (socket says 172.21.15.99:60526) 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.349+0000 7faa83e00700 1 -- 172.21.15.99:0/1124464174 learned_addr learned my addr 172.21.15.99:0/1124464174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.349+0000 7faa83e00700 1 -- 172.21.15.99:0/1124464174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8410f290 con 0x7faa84081a20 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.349+0000 7faa83e00700 1 --2- 172.21.15.99:0/1124464174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa740098a0 tx=0x7faa74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c7c26f8e3f04d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.349+0000 7faa82a00700 1 -- 172.21.15.99:0/1124464174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa74004160 con 0x7faa84081a20 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa82a00700 1 -- 172.21.15.99:0/1124464174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa74004300 con 0x7faa84081a20 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa82a00700 1 -- 172.21.15.99:0/1124464174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa74011b20 con 0x7faa84081a20 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 -- 172.21.15.99:0/1124464174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 msgr2=0x7faa84081e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 --2- 172.21.15.99:0/1124464174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa740098a0 tx=0x7faa74009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 -- 172.21.15.99:0/1124464174 shutdown_connections 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 --2- 172.21.15.99:0/1124464174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa84081e40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 -- 172.21.15.99:0/1124464174 >> 172.21.15.99:0/1124464174 conn(0x7faa8407d0d0 msgr2=0x7faa8407f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 -- 172.21.15.99:0/1124464174 shutdown_connections 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.350+0000 7faa8ac00700 1 -- 172.21.15.99:0/1124464174 wait complete. 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.351+0000 7faa8ac00700 1 Processor -- start 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa8ac00700 1 -- start start 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa8ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa8ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa84112640 con 0x7faa84081a20 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa83e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa83e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60534/0 (socket says 172.21.15.99:60534) 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa83e00700 1 -- 172.21.15.99:0/2631152670 learned_addr learned my addr 172.21.15.99:0/2631152670 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:14.731 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.352+0000 7faa83e00700 1 -- 172.21.15.99:0/2631152670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa841a24f0 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa83e00700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa740117e0 tx=0x7faa74011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa74003bf0 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa7401a910 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa8ac00700 1 -- 172.21.15.99:0/2631152670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa841a2780 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa740249a0 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.353+0000 7faa8ac00700 1 -- 172.21.15.99:0/2631152670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa841a2be0 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.354+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7faa74022020 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.355+0000 7faa80c00700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 0x7faa6803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.355+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa840637c0 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.355+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa7404d420 con 0x7faa84081a20 2024-03-23T23:33:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.355+0000 7faa83400700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 0x7faa6803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.356+0000 7faa83400700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 0x7faa6803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa70006fd0 tx=0x7faa70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.361+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa74011e20 con 0x7faa84081a20 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.599+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7faa84062ee0 con 0x7faa84081a20 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.599+0000 7faa80c00700 1 -- 172.21.15.99:0/2631152670 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7faa74004300 con 0x7faa84081a20 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 msgr2=0x7faa6803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 0x7faa6803b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa70006fd0 tx=0x7faa70006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 msgr2=0x7faa841a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa740117e0 tx=0x7faa74011c10 comp rx=0 tx=0).stop 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 shutdown_connections 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa68039020 0x7faa6803b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:14.733 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 --2- 172.21.15.99:0/2631152670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa84081a20 0x7faa841a1fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:14.734 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 >> 172.21.15.99:0/2631152670 conn(0x7faa8407d0d0 msgr2=0x7faa8407db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:14.734 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 shutdown_connections 2024-03-23T23:33:14.734 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:13.602+0000 7faa7a000700 1 -- 172.21.15.99:0/2631152670 wait complete. 2024-03-23T23:33:14.758 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T23:33:14.758 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-23T23:33:14.782 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T23:33:14.783 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-23T23:33:14.786 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-23T23:33:14.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph Dashboard is now available at: 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout: URL: https://smithi099.front.sepia.ceph.com:8443/ 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout: User: admin 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout: Password: 2rsdqx67ct 2024-03-23T23:33:14.790 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:14.791 INFO:teuthology.orchestra.run.smithi099.stdout:Saving cluster configuration to /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config directory 2024-03-23T23:33:14.793 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling autotune for osd_memory_target 2024-03-23T23:33:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.169+0000 7f9766200700 1 Processor -- start 2024-03-23T23:33:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.170+0000 7f9766200700 1 -- start start 2024-03-23T23:33:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.170+0000 7f9766200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.170+0000 7f9766200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976007f270 con 0x7f976007e880 2024-03-23T23:33:17.292 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.171+0000 7f975f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:17.292 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.171+0000 7f975f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60538/0 (socket says 172.21.15.99:60538) 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.171+0000 7f975f400700 1 -- 172.21.15.99:0/2347221916 learned_addr learned my addr 172.21.15.99:0/2347221916 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.172+0000 7f975f400700 1 -- 172.21.15.99:0/2347221916 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976007fad0 con 0x7f976007e880 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.172+0000 7f975f400700 1 --2- 172.21.15.99:0/2347221916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9750009e80 tx=0x7f975000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe5c7a74dc2a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.172+0000 7f975ea00700 1 -- 172.21.15.99:0/2347221916 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9750017070 con 0x7f976007e880 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.172+0000 7f975ea00700 1 -- 172.21.15.99:0/2347221916 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975000b8c0 con 0x7f976007e880 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f975ea00700 1 -- 172.21.15.99:0/2347221916 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97500045c0 con 0x7f976007e880 2024-03-23T23:33:17.293 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f9766200700 1 -- 172.21.15.99:0/2347221916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 msgr2=0x7f976007eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:17.296 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f9766200700 1 --2- 172.21.15.99:0/2347221916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9750009e80 tx=0x7f975000b330 comp rx=0 tx=0).stop 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f9766200700 1 -- 172.21.15.99:0/2347221916 shutdown_connections 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f9766200700 1 --2- 172.21.15.99:0/2347221916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f976007eca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.173+0000 7f9766200700 1 -- 172.21.15.99:0/2347221916 >> 172.21.15.99:0/2347221916 conn(0x7f976007d0d0 msgr2=0x7f976007d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.174+0000 7f9766200700 1 -- 172.21.15.99:0/2347221916 shutdown_connections 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.174+0000 7f9766200700 1 -- 172.21.15.99:0/2347221916 wait complete. 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.174+0000 7f9766200700 1 Processor -- start 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f9766200700 1 -- start start 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f9766200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f9766200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9760080e90 con 0x7f976007e880 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f975f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f975f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60546/0 (socket says 172.21.15.99:60546) 2024-03-23T23:33:17.297 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.175+0000 7f975f400700 1 -- 172.21.15.99:0/1650004855 learned_addr learned my addr 172.21.15.99:0/1650004855 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f975f400700 1 -- 172.21.15.99:0/1650004855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97601a2570 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f975f400700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f975000b7b0 tx=0x7f975000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9750020020 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9750003d90 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97601a2800 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9750017040 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.176+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a54f0 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.178+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f9750023020 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.178+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97600637c0 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.178+0000 7f975cc00700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 0x7f974403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.178+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f975000f050 con 0x7f976007e880 2024-03-23T23:33:17.298 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.178+0000 7f9756a00700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 0x7f974403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.179+0000 7f9756a00700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 0x7f974403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9748006fd0 tx=0x7f9748006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.183+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f975002a020 con 0x7f976007e880 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.401+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f976002dcf0 con 0x7f976007e880 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.402+0000 7f975cc00700 1 -- 172.21.15.99:0/1650004855 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9750012a50 con 0x7f976007e880 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 msgr2=0x7f974403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 0x7f974403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9748006fd0 tx=0x7f9748006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 msgr2=0x7f97601a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f975000b7b0 tx=0x7f975000bd50 comp rx=0 tx=0).stop 2024-03-23T23:33:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 shutdown_connections 2024-03-23T23:33:17.300 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9744038fc0 0x7f974403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:17.300 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 --2- 172.21.15.99:0/1650004855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f976007e880 0x7f97601a2030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:17.300 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 >> 172.21.15.99:0/1650004855 conn(0x7f976007d0d0 msgr2=0x7f9760198cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:17.300 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 shutdown_connections 2024-03-23T23:33:17.300 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:16.405+0000 7f9766200700 1 -- 172.21.15.99:0/1650004855 wait complete. 2024-03-23T23:33:19.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.014+0000 7f6b30c00700 1 Processor -- start 2024-03-23T23:33:19.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.015+0000 7f6b30c00700 1 -- start start 2024-03-23T23:33:19.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.015+0000 7f6b30c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:19.027 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.015+0000 7f6b30c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c082410 con 0x7f6b2c081a20 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.015+0000 7f6b29e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.016+0000 7f6b29e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60558/0 (socket says 172.21.15.99:60558) 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.016+0000 7f6b29e00700 1 -- 172.21.15.99:0/1206752592 learned_addr learned my addr 172.21.15.99:0/1206752592 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.016+0000 7f6b29e00700 1 -- 172.21.15.99:0/1206752592 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c10f290 con 0x7f6b2c081a20 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.016+0000 7f6b29e00700 1 --2- 172.21.15.99:0/1206752592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0098a0 tx=0x7f6b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a153073c34e52 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b29400700 1 -- 172.21.15.99:0/1206752592 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c004160 con 0x7f6b2c081a20 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b29400700 1 -- 172.21.15.99:0/1206752592 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b1c004300 con 0x7f6b2c081a20 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b29400700 1 -- 172.21.15.99:0/1206752592 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c011b20 con 0x7f6b2c081a20 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b30c00700 1 -- 172.21.15.99:0/1206752592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 msgr2=0x7f6b2c081e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1206752592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0098a0 tx=0x7f6b1c009bb0 comp rx=0 tx=0).stop 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b30c00700 1 -- 172.21.15.99:0/1206752592 shutdown_connections 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.017+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1206752592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c081e40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.018+0000 7f6b30c00700 1 -- 172.21.15.99:0/1206752592 >> 172.21.15.99:0/1206752592 conn(0x7f6b2c07d0d0 msgr2=0x7f6b2c07f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.018+0000 7f6b30c00700 1 -- 172.21.15.99:0/1206752592 shutdown_connections 2024-03-23T23:33:19.028 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.018+0000 7f6b30c00700 1 -- 172.21.15.99:0/1206752592 wait complete. 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b30c00700 1 Processor -- start 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b30c00700 1 -- start start 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b30c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b30c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c112640 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b29e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b29e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60572/0 (socket says 172.21.15.99:60572) 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.019+0000 7f6b29e00700 1 -- 172.21.15.99:0/1763160338 learned_addr learned my addr 172.21.15.99:0/1763160338 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b29e00700 1 -- 172.21.15.99:0/1763160338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c1a24f0 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b29e00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0117e0 tx=0x7f6b1c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c003bf0 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b1c01a910 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c1a2780 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c024970 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.020+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c1a2be0 con 0x7f6b2c081a20 2024-03-23T23:33:19.029 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.021+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6b1c022020 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.021+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0637c0 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.022+0000 7f6b22a00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 0x7f6b1003b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.022+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b1c04d3b0 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.022+0000 7f6b23e00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 0x7f6b1003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.023+0000 7f6b23e00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 0x7f6b1003b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006fd0 tx=0x7f6b14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.028+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b1c010380 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.362+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6b2c1a55b0 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.366+0000 7f6b22a00700 1 -- 172.21.15.99:0/1763160338 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6b1c017ca0 con 0x7f6b2c081a20 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 msgr2=0x7f6b1003b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 0x7f6b1003b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006fd0 tx=0x7f6b14006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 msgr2=0x7f6b2c1a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0117e0 tx=0x7f6b1c011ba0 comp rx=0 tx=0).stop 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 shutdown_connections 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6b10039070 0x7f6b1003b530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:19.030 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 --2- 172.21.15.99:0/1763160338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b2c081a20 0x7f6b2c1a1fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 >> 172.21.15.99:0/1763160338 conn(0x7f6b2c07d0d0 msgr2=0x7f6b2c07db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 shutdown_connections 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-23T23:33:18.369+0000 7f6b30c00700 1 -- 172.21.15.99:0/1763160338 wait complete. 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout:Or, if you are only running a single cluster on this host: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: ceph telemetry on 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout:For more information see: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:19.032 INFO:teuthology.orchestra.run.smithi099.stdout:Bootstrap complete. 2024-03-23T23:33:19.150 INFO:tasks.cephadm:Fetching config... 2024-03-23T23:33:19.150 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:33:19.150 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-23T23:33:19.237 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-23T23:33:19.238 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:33:19.238 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-23T23:33:19.323 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-23T23:33:19.323 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:33:19.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-23T23:33:19.469 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-23T23:33:19.469 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:33:19.469 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-23T23:33:19.521 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-23T23:33:19.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T23:33:19.594 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:19 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1763160338' entity='client.admin' 2024-03-23T23:33:19.629 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:33:19.697 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T23:33:19.743 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:33:19.758 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T23:33:19.806 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWJOXGfwg8WcTzRoqjt5zVXlscA4GWxtW741tnYdZyN1QvNdM1Mjb5n9t5RA1fLe+z3PDvL+fwbs+HEYPXsJIv1S2I7LUeAM/OHgCClk6qNPo1YK3D6s4EmiIucIUcM9RdJT8z+gF+/56JrPHKo8jANr0rqdKH25TLPuhry0QWp5Jywax/Sbr7s903IN1lJ3vMPYngsBGg1Fs7UCuZ+6hyKl2s9xD1qiibczENdy95yoa9lTkvRdV/0Kj7IFY58QEewxWgy6TMW5JWUMC5VBhp9h9ZGcvuV1EmDcAhijQKPuVp+W4UAkP/l7KVqZ0Dis1BlRHYoUtjxW/axC9WuPIRRbjqly0p99KeKZ0qXJjBc6uNkDf1KXMMZmIkk1ss+8TjEqOiyy/M2iZRClDvidjcLgfXWFp4YeBGk8Bv29enEP5EPdstJ6mfXTf/N6n1AufBxKMIGOtjKMa6x7ohynyUSN0/J6LvgwkZYKXtV0jF0m7QgISMLtTRJwb1aZKU/U= ceph-5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:33:19.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-23T23:33:20.241 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:23.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:23.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:23.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:33:23.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:23.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:23.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 -- 172.21.15.99:0/3187896727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 msgr2=0x7fd758102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 --2- 172.21.15.99:0/3187896727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758102010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd748009910 tx=0x7fd748009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 -- 172.21.15.99:0/3187896727 shutdown_connections 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 --2- 172.21.15.99:0/3187896727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758102010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 -- 172.21.15.99:0/3187896727 >> 172.21.15.99:0/3187896727 conn(0x7fd7580fd1b0 msgr2=0x7fd7580ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 -- 172.21.15.99:0/3187896727 shutdown_connections 2024-03-23T23:33:23.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.436+0000 7fd75ca00700 1 -- 172.21.15.99:0/3187896727 wait complete. 2024-03-23T23:33:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd75ca00700 1 Processor -- start 2024-03-23T23:33:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd75ca00700 1 -- start start 2024-03-23T23:33:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd75ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd75ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758107c80 con 0x7fd758101bf0 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd755e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd755e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43676/0 (socket says 172.21.15.99:43676) 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.438+0000 7fd755e00700 1 -- 172.21.15.99:0/4237572921 learned_addr learned my addr 172.21.15.99:0/4237572921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd755e00700 1 -- 172.21.15.99:0/4237572921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758076390 con 0x7fd758101bf0 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd755e00700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd748005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:23.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd748027070 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd74800be40 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758074580 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.439+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74800f460 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.440+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758074ac0 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.440+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758108400 con 0x7fd758101bf0 2024-03-23T23:33:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.441+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd74802a070 con 0x7fd758101bf0 2024-03-23T23:33:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.441+0000 7fd74f400700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 0x7fd73c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.441+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd74801e030 con 0x7fd758101bf0 2024-03-23T23:33:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.441+0000 7fd74d400700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 0x7fd73c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.442+0000 7fd74d400700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 0x7fd73c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:23.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.447+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd748013070 con 0x7fd758101bf0 2024-03-23T23:33:23.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.670+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd758190700 con 0x7fd758101bf0 2024-03-23T23:33:23.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.676+0000 7fd74f400700 1 -- 172.21.15.99:0/4237572921 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd7480310c0 con 0x7fd758101bf0 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.678+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 msgr2=0x7fd73c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.678+0000 7fd75ca00700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 0x7fd73c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.678+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 msgr2=0x7fd758075e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.678+0000 7fd75ca00700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd748006010 tx=0x7fd748005050 comp rx=0 tx=0).stop 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 shutdown_connections 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd73c038fc0 0x7fd73c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 --2- 172.21.15.99:0/4237572921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd758101bf0 0x7fd758075e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 >> 172.21.15.99:0/4237572921 conn(0x7fd7580fd1b0 msgr2=0x7fd7580fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 shutdown_connections 2024-03-23T23:33:23.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:23.679+0000 7fd75ca00700 1 -- 172.21.15.99:0/4237572921 wait complete. 2024-03-23T23:33:24.434 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-23T23:33:24.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-23T23:33:24.892 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:25.039 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:24 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4237572921' entity='client.admin' 2024-03-23T23:33:25.040 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:25.040 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:27.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 -- 172.21.15.99:0/3783437786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 msgr2=0x7f8f940745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3783437786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f940745c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009910 tx=0x7f8f84009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 -- 172.21.15.99:0/3783437786 shutdown_connections 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3783437786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f940745c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 -- 172.21.15.99:0/3783437786 >> 172.21.15.99:0/3783437786 conn(0x7f8f940fd160 msgr2=0x7f8f940ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 -- 172.21.15.99:0/3783437786 shutdown_connections 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.209+0000 7f8f98e00700 1 -- 172.21.15.99:0/3783437786 wait complete. 2024-03-23T23:33:27.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.210+0000 7f8f98e00700 1 Processor -- start 2024-03-23T23:33:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.210+0000 7f8f98e00700 1 -- start start 2024-03-23T23:33:27.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.210+0000 7f8f98e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.210+0000 7f8f98e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94109e40 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.210+0000 7f8f91e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f91e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43692/0 (socket says 172.21.15.99:43692) 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f91e00700 1 -- 172.21.15.99:0/3735204109 learned_addr learned my addr 172.21.15.99:0/3735204109 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f91e00700 1 -- 172.21.15.99:0/3735204109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f941999c0 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f91e00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005950 tx=0x7f8f84005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f84026070 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f8400bc30 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f84003b30 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.211+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f94199c50 con 0x7f8f94076160 2024-03-23T23:33:27.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.212+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9419c940 con 0x7f8f94076160 2024-03-23T23:33:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.213+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f8f84029070 con 0x7f8f94076160 2024-03-23T23:33:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.213+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9410a5c0 con 0x7f8f94076160 2024-03-23T23:33:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.213+0000 7f8f8aa00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 0x7f8f7803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:27.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.213+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f8401e070 con 0x7f8f94076160 2024-03-23T23:33:27.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.213+0000 7f8f91400700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 0x7f8f7803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:27.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.214+0000 7f8f91400700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 0x7f8f7803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:27.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.219+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f84013070 con 0x7f8f94076160 2024-03-23T23:33:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.436+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f9402db40 con 0x7f8f78039020 2024-03-23T23:33:27.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.442+0000 7f8f8aa00700 1 -- 172.21.15.99:0/3735204109 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8f9402db40 con 0x7f8f78039020 2024-03-23T23:33:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 msgr2=0x7f8f7803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 0x7f8f7803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 msgr2=0x7f8f94199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005950 tx=0x7f8f84005030 comp rx=0 tx=0).stop 2024-03-23T23:33:27.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 shutdown_connections 2024-03-23T23:33:27.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8f78039020 0x7f8f7803b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:27.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 --2- 172.21.15.99:0/3735204109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f94076160 0x7f8f94199480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:27.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 >> 172.21.15.99:0/3735204109 conn(0x7f8f940fd160 msgr2=0x7f8f940fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:27.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.445+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 shutdown_connections 2024-03-23T23:33:27.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:27.446+0000 7f8f98e00700 1 -- 172.21.15.99:0/3735204109 wait complete. 2024-03-23T23:33:27.860 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:27 smithi099 ceph-mon[31715]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:27.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:27.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:27.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:33:27.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:33:28.353 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-03-23T23:33:28.354 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:33:28.354 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-03-23T23:33:28.372 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:33:28.372 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:28.430 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-03-23T23:33:28.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch host add smithi149 2024-03-23T23:33:28.837 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:28.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:28 smithi099 ceph-mon[31715]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:33:28.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:28 smithi099 ceph-mon[31715]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-23T23:33:30.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:29 smithi099 ceph-mon[31715]: Updating smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:33:30.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:29 smithi099 ceph-mon[31715]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:30.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:29 smithi099 ceph-mon[31715]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 -- 172.21.15.99:0/3827590237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 msgr2=0x7f3178106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 --2- 172.21.15.99:0/3827590237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f3178106e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3168009910 tx=0x7f3168009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 -- 172.21.15.99:0/3827590237 shutdown_connections 2024-03-23T23:33:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 --2- 172.21.15.99:0/3827590237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f3178106e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 -- 172.21.15.99:0/3827590237 >> 172.21.15.99:0/3827590237 conn(0x7f31780776d0 msgr2=0x7f31780fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 -- 172.21.15.99:0/3827590237 shutdown_connections 2024-03-23T23:33:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.995+0000 7f317f200700 1 -- 172.21.15.99:0/3827590237 wait complete. 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.996+0000 7f317f200700 1 Processor -- start 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.996+0000 7f317f200700 1 -- start start 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f317f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f317f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3178109f90 con 0x7f3178106a60 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f3177e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f3177e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58008/0 (socket says 172.21.15.99:58008) 2024-03-23T23:33:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f3177e00700 1 -- 172.21.15.99:0/2130588595 learned_addr learned my addr 172.21.15.99:0/2130588595 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.997+0000 7f3177e00700 1 -- 172.21.15.99:0/2130588595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317819faf0 con 0x7f3178106a60 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f3177e00700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f3168005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3168026070 con 0x7f3178106a60 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f316800bbd0 con 0x7f3178106a60 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3168003910 con 0x7f3178106a60 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317819fde0 con 0x7f3178106a60 2024-03-23T23:33:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.998+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31781998a0 con 0x7f3178106a60 2024-03-23T23:33:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.999+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31781021a0 con 0x7f3178106a60 2024-03-23T23:33:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:30.999+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f3168029070 con 0x7f3178106a60 2024-03-23T23:33:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.000+0000 7f3174c00700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 0x7f315c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.000+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f316801e070 con 0x7f3178106a60 2024-03-23T23:33:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.000+0000 7f3177400700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 0x7f315c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.001+0000 7f3177400700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 0x7f315c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:31.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.005+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3168013070 con 0x7f3178106a60 2024-03-23T23:33:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:30 smithi099 ceph-mon[31715]: Updating smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T23:33:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:31.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:31.234+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7f317819abe0 con 0x7f315c039020 2024-03-23T23:33:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:31 smithi099 ceph-mon[31715]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:33.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:32 smithi099 ceph-mon[31715]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:33:34.360 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:33 smithi099 ceph-mon[31715]: Deploying cephadm binary to smithi149 2024-03-23T23:33:34.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:33 smithi099 ceph-mon[31715]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.897+0000 7f3174c00700 1 -- 172.21.15.99:0/2130588595 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f317819abe0 con 0x7f315c039020 2024-03-23T23:33:35.898 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-03-23T23:33:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 msgr2=0x7f315c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 0x7f315c03b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 msgr2=0x7f317819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3168006010 tx=0x7f3168005030 comp rx=0 tx=0).stop 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 shutdown_connections 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f315c039020 0x7f315c03b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 --2- 172.21.15.99:0/2130588595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3178106a60 0x7f317819f5b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 >> 172.21.15.99:0/2130588595 conn(0x7f31780776d0 msgr2=0x7f3178108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.901+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 shutdown_connections 2024-03-23T23:33:35.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:35.902+0000 7f317f200700 1 -- 172.21.15.99:0/2130588595 wait complete. 2024-03-23T23:33:36.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:35 smithi099 ceph-mon[31715]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:36.224 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:36.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T23:33:37.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:36 smithi099 ceph-mon[31715]: Added host smithi149 2024-03-23T23:33:37.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:36 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:37.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:36 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:37.374 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:38.060 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:37 smithi099 ceph-mon[31715]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:38.061 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 -- 172.21.15.99:0/53917966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 msgr2=0x7f968c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 --2- 172.21.15.99:0/53917966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c0749a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f967c009910 tx=0x7f967c009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 -- 172.21.15.99:0/53917966 shutdown_connections 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 --2- 172.21.15.99:0/53917966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c0749a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 -- 172.21.15.99:0/53917966 >> 172.21.15.99:0/53917966 conn(0x7f968c0fd670 msgr2=0x7f968c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 -- 172.21.15.99:0/53917966 shutdown_connections 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.656+0000 7f9691200700 1 -- 172.21.15.99:0/53917966 wait complete. 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.657+0000 7f9691200700 1 Processor -- start 2024-03-23T23:33:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9691200700 1 -- start start 2024-03-23T23:33:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9691200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9691200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c10e410 con 0x7f968c0745c0 2024-03-23T23:33:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9689e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9689e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60638/0 (socket says 172.21.15.99:60638) 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.658+0000 7f9689e00700 1 -- 172.21.15.99:0/2825928549 learned_addr learned my addr 172.21.15.99:0/2825928549 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9689e00700 1 -- 172.21.15.99:0/2825928549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c19fb70 con 0x7f968c0745c0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9689e00700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f967c006010 tx=0x7f967c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f967c026070 con 0x7f968c0745c0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f967c00bc30 con 0x7f968c0745c0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f967c0039a0 con 0x7f968c0745c0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c19fe00 con 0x7f968c0745c0 2024-03-23T23:33:39.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.659+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c199700 con 0x7f968c0745c0 2024-03-23T23:33:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.661+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c075c70 con 0x7f968c0745c0 2024-03-23T23:33:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.661+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f967c029070 con 0x7f968c0745c0 2024-03-23T23:33:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.661+0000 7f9682a00700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 0x7f967003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.661+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f967c01e070 con 0x7f968c0745c0 2024-03-23T23:33:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.662+0000 7f9683e00700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 0x7f967003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:39.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.662+0000 7f9683e00700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 0x7f967003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9674006fd0 tx=0x7f9674006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.667+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f967c013070 con 0x7f968c0745c0 2024-03-23T23:33:39.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:39.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:39 smithi099 ceph-mon[31715]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.899+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f968c19ac30 con 0x7f9670039020 2024-03-23T23:33:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.901+0000 7f9682a00700 1 -- 172.21.15.99:0/2825928549 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f968c19ac30 con 0x7f9670039020 2024-03-23T23:33:39.900 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:39.901 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-03-23T23:33:39.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 msgr2=0x7f967003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 0x7f967003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9674006fd0 tx=0x7f9674006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 msgr2=0x7f968c19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f967c006010 tx=0x7f967c005030 comp rx=0 tx=0).stop 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 shutdown_connections 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9670039020 0x7f967003b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 --2- 172.21.15.99:0/2825928549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c0745c0 0x7f968c19f630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.903+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 >> 172.21.15.99:0/2825928549 conn(0x7f968c0fd670 msgr2=0x7f968c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.904+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 shutdown_connections 2024-03-23T23:33:39.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:39.904+0000 7f9691200700 1 -- 172.21.15.99:0/2825928549 wait complete. 2024-03-23T23:33:40.779 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-03-23T23:33:40.780 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:33:40.780 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-03-23T23:33:40.798 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:33:40.798 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:40.856 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-03-23T23:33:40.856 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch host add smithi174 2024-03-23T23:33:40.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:40 smithi099 ceph-mon[31715]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:33:41.275 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:41.674 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:41 smithi099 ceph-mon[31715]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:43.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:43.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:33:43.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:42 smithi099 ceph-mon[31715]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-23T23:33:43.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 -- 172.21.15.99:0/2383275816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 msgr2=0x7f91c41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2383275816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41087d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009910 tx=0x7f91b4009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 -- 172.21.15.99:0/2383275816 shutdown_connections 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2383275816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41087d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 -- 172.21.15.99:0/2383275816 >> 172.21.15.99:0/2383275816 conn(0x7f91c4077930 msgr2=0x7f91c4079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 -- 172.21.15.99:0/2383275816 shutdown_connections 2024-03-23T23:33:43.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.728+0000 7f91c8c00700 1 -- 172.21.15.99:0/2383275816 wait complete. 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.729+0000 7f91c8c00700 1 Processor -- start 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c8c00700 1 -- start start 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c8c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c41127c0 con 0x7f91c41083f0 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60650/0 (socket says 172.21.15.99:60650) 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.730+0000 7f91c1e00700 1 -- 172.21.15.99:0/2684002839 learned_addr learned my addr 172.21.15.99:0/2684002839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91c1e00700 1 -- 172.21.15.99:0/2684002839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c41a3f20 con 0x7f91c41083f0 2024-03-23T23:33:43.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91c1e00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:43.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b4026070 con 0x7f91c41083f0 2024-03-23T23:33:43.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91b400bbd0 con 0x7f91c41083f0 2024-03-23T23:33:43.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b4003910 con 0x7f91c41083f0 2024-03-23T23:33:43.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c41a4210 con 0x7f91c41083f0 2024-03-23T23:33:43.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.731+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c419dbe0 con 0x7f91c41083f0 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.732+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f91b4029070 con 0x7f91c41083f0 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.732+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c41098d0 con 0x7f91c41083f0 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.733+0000 7f91baa00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 0x7f91a803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.733+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91b401e070 con 0x7f91c41083f0 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.733+0000 7f91c1400700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 0x7f91a803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:43.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.734+0000 7f91c1400700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 0x7f91a803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006fd0 tx=0x7f91b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:43.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.739+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91b4013070 con 0x7f91c41083f0 2024-03-23T23:33:43.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:43.964+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) v1 -- 0x7f91c419efe0 con 0x7f91a8039020 2024-03-23T23:33:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: Updating smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:33:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: Updating smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T23:33:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:44.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:44.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:45 smithi099 ceph-mon[31715]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:33:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:45 smithi099 ceph-mon[31715]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:45 smithi099 ceph-mon[31715]: Deploying cephadm binary to smithi174 2024-03-23T23:33:48.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:47 smithi099 ceph-mon[31715]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:48.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.652+0000 7f91baa00700 1 -- 172.21.15.99:0/2684002839 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f91c419efe0 con 0x7f91a8039020 2024-03-23T23:33:48.653 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-03-23T23:33:48.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 msgr2=0x7f91a803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:48.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 0x7f91a803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006fd0 tx=0x7f91b0006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 msgr2=0x7f91c41a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91b4006010 tx=0x7f91b4005030 comp rx=0 tx=0).stop 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 shutdown_connections 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f91a8039020 0x7f91a803b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 --2- 172.21.15.99:0/2684002839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91c41083f0 0x7f91c41a39e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.656+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 >> 172.21.15.99:0/2684002839 conn(0x7f91c4077930 msgr2=0x7f91c4109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.657+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 shutdown_connections 2024-03-23T23:33:48.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:48.657+0000 7f91c8c00700 1 -- 172.21.15.99:0/2684002839 wait complete. 2024-03-23T23:33:49.461 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T23:33:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:49 smithi099 ceph-mon[31715]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:49 smithi099 ceph-mon[31715]: Added host smithi174 2024-03-23T23:33:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:33:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:49.895 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:51.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:52.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 -- 172.21.15.99:0/1333222733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 msgr2=0x7fc7b81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:52.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 --2- 172.21.15.99:0/1333222733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81087d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009910 tx=0x7fc7a8009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:52.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 -- 172.21.15.99:0/1333222733 shutdown_connections 2024-03-23T23:33:52.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 --2- 172.21.15.99:0/1333222733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81087d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:52.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 -- 172.21.15.99:0/1333222733 >> 172.21.15.99:0/1333222733 conn(0x7fc7b8077930 msgr2=0x7fc7b8079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:52.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 -- 172.21.15.99:0/1333222733 shutdown_connections 2024-03-23T23:33:52.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.009+0000 7fc7bec00700 1 -- 172.21.15.99:0/1333222733 wait complete. 2024-03-23T23:33:52.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.010+0000 7fc7bec00700 1 Processor -- start 2024-03-23T23:33:52.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.010+0000 7fc7bec00700 1 -- start start 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7bec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7bec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b81127c0 con 0x7fc7b81083f0 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44616/0 (socket says 172.21.15.99:44616) 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7b7e00700 1 -- 172.21.15.99:0/461702795 learned_addr learned my addr 172.21.15.99:0/461702795 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.011+0000 7fc7b7e00700 1 -- 172.21.15.99:0/461702795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b81a3e30 con 0x7fc7b81083f0 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7b7e00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006010 tx=0x7fc7a8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7a8026070 con 0x7fc7b81083f0 2024-03-23T23:33:52.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7a800bbd0 con 0x7fc7b81083f0 2024-03-23T23:33:52.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b81a4120 con 0x7fc7b81083f0 2024-03-23T23:33:52.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7a8003910 con 0x7fc7b81083f0 2024-03-23T23:33:52.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.012+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b819dab0 con 0x7fc7b81083f0 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.013+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fc7a8029070 con 0x7fc7b81083f0 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.013+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b81098d0 con 0x7fc7b81083f0 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.014+0000 7fc7b4c00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 0x7fc79c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.014+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc7a801e070 con 0x7fc7b81083f0 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.014+0000 7fc7b7400700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 0x7fc79c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:52.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.015+0000 7fc7b7400700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 0x7fc79c03b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4006fd0 tx=0x7fc7a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.020+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc7a8013070 con 0x7fc7b81083f0 2024-03-23T23:33:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:51 smithi099 ceph-mon[31715]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:52.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.249+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7b819f050 con 0x7fc79c038fc0 2024-03-23T23:33:52.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.251+0000 7fc7b4c00700 1 -- 172.21.15.99:0/461702795 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fc7b819f050 con 0x7fc79c038fc0 2024-03-23T23:33:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:33:52.252 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 msgr2=0x7fc79c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 0x7fc79c03b480 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4006fd0 tx=0x7fc7a4006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 msgr2=0x7fc7b81a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006010 tx=0x7fc7a8005030 comp rx=0 tx=0).stop 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 shutdown_connections 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc79c038fc0 0x7fc79c03b480 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 --2- 172.21.15.99:0/461702795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7b81083f0 0x7fc7b81a38f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.254+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 >> 172.21.15.99:0/461702795 conn(0x7fc7b8077930 msgr2=0x7fc7b8109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.255+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 shutdown_connections 2024-03-23T23:33:52.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:52.255+0000 7fc7bec00700 1 -- 172.21.15.99:0/461702795 wait complete. 2024-03-23T23:33:53.298 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-23T23:33:53.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-23T23:33:53.736 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:33:54.084 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:53 smithi099 ceph-mon[31715]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:33:54.084 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:53 smithi099 ceph-mon[31715]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:55.712 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:55.712 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:55.712 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:55.712 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:55.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:55 smithi099 ceph-mon[31715]: Updating smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 -- 172.21.15.99:0/1013533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 msgr2=0x7f38ac1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 --2- 172.21.15.99:0/1013533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac1021a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f389c00b0a0 tx=0x7f389c00b3b0 comp rx=0 tx=0).stop 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 -- 172.21.15.99:0/1013533417 shutdown_connections 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 --2- 172.21.15.99:0/1013533417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac1021a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 -- 172.21.15.99:0/1013533417 >> 172.21.15.99:0/1013533417 conn(0x7f38ac0fd670 msgr2=0x7f38ac0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 -- 172.21.15.99:0/1013533417 shutdown_connections 2024-03-23T23:33:55.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.899+0000 7f38b2a00700 1 -- 172.21.15.99:0/1013533417 wait complete. 2024-03-23T23:33:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.901+0000 7f38b2a00700 1 Processor -- start 2024-03-23T23:33:55.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.901+0000 7f38b2a00700 1 -- start start 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38b2a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10c190 con 0x7f38ac101dc0 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44620/0 (socket says 172.21.15.99:44620) 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38abe00700 1 -- 172.21.15.99:0/611708096 learned_addr learned my addr 172.21.15.99:0/611708096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:33:55.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38abe00700 1 -- 172.21.15.99:0/611708096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac076380 con 0x7f38ac101dc0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.902+0000 7f38abe00700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f389c0093e0 tx=0x7f389c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.903+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c017040 con 0x7f38ac101dc0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.903+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f389c022070 con 0x7f38ac101dc0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.903+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac074a30 con 0x7f38ac101dc0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.903+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c01d440 con 0x7f38ac101dc0 2024-03-23T23:33:55.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.903+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac074ef0 con 0x7f38ac101dc0 2024-03-23T23:33:55.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.904+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac1032a0 con 0x7f38ac101dc0 2024-03-23T23:33:55.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.904+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f389c004000 con 0x7f38ac101dc0 2024-03-23T23:33:55.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.905+0000 7f38a9600700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 0x7f389003b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:55.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.905+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f389c00f050 con 0x7f38ac101dc0 2024-03-23T23:33:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.905+0000 7f38a3400700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 0x7f389003b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:55.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.905+0000 7f38a3400700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 0x7f389003b4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:55.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:55.910+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f389c007e20 con 0x7f38ac101dc0 2024-03-23T23:33:56.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.141+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f38ac075a60 con 0x7f38ac101dc0 2024-03-23T23:33:56.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.624+0000 7f38a9600700 1 -- 172.21.15.99:0/611708096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f389c04cd50 con 0x7f38ac101dc0 2024-03-23T23:33:56.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.627+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 msgr2=0x7f389003b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:56.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.627+0000 7f38b2a00700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 0x7f389003b4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:56.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 msgr2=0x7f38ac074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:56.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f389c0093e0 tx=0x7f389c003ce0 comp rx=0 tx=0).stop 2024-03-23T23:33:56.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 shutdown_connections 2024-03-23T23:33:56.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3890039010 0x7f389003b4d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:56.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 --2- 172.21.15.99:0/611708096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ac101dc0 0x7f38ac074490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:56.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 >> 172.21.15.99:0/611708096 conn(0x7f38ac0fd670 msgr2=0x7f38ac103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:56.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 shutdown_connections 2024-03-23T23:33:56.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:33:56.628+0000 7f38b2a00700 1 -- 172.21.15.99:0/611708096 wait complete. 2024-03-23T23:33:56.630 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-03-23T23:33:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:33:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: Updating smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T23:33:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:33:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:56 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/611708096' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T23:33:57.560 INFO:tasks.cephadm:Adding mon.a on smithi099 2024-03-23T23:33:57.560 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-03-23T23:33:57.560 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-03-23T23:33:57.560 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi099:172.21.15.99=a;smithi149:172.21.15.149=b;smithi174:172.21.15.174=c' 2024-03-23T23:33:57.798 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:33:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:57 smithi099 ceph-mon[31715]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:33:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:57 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/611708096' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T23:33:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:57 smithi099 ceph-mon[31715]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 -- 172.21.15.174:0/3435356176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 msgr2=0x7fd1c8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 --2- 172.21.15.174:0/3435356176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c8106e40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009910 tx=0x7fd1b8009c20 comp rx=0 tx=0).stop 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 -- 172.21.15.174:0/3435356176 shutdown_connections 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 --2- 172.21.15.174:0/3435356176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c8106e40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 -- 172.21.15.174:0/3435356176 >> 172.21.15.174:0/3435356176 conn(0x7fd1c80fc550 msgr2=0x7fd1c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 -- 172.21.15.174:0/3435356176 shutdown_connections 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.228+0000 7fd1cd333700 1 -- 172.21.15.174:0/3435356176 wait complete. 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.229+0000 7fd1cd333700 1 Processor -- start 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.229+0000 7fd1cd333700 1 -- start start 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.229+0000 7fd1cd333700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.229+0000 7fd1cd333700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c810b0a0 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.230+0000 7fd1c6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.230+0000 7fd1c6ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.174:41228/0 (socket says 172.21.15.174:41228) 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.230+0000 7fd1c6ffd700 1 -- 172.21.15.174:0/1905027253 learned_addr learned my addr 172.21.15.174:0/1905027253 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.230+0000 7fd1c6ffd700 1 -- 172.21.15.174:0/1905027253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8198a30 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1c6ffd700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005850 tx=0x7fd1b8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b8026070 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c8198d20 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c81991e0 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b800bbd0 con 0x7fd1c8106a60 2024-03-23T23:33:59.231 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.231+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b80039b0 con 0x7fd1c8106a60 2024-03-23T23:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.232+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c81021a0 con 0x7fd1c8106a60 2024-03-23T23:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.232+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd1b8029070 con 0x7fd1c8106a60 2024-03-23T23:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.232+0000 7fd1affff700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 0x7fd1b003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:33:59.232 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.232+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1b801e070 con 0x7fd1c8106a60 2024-03-23T23:33:59.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.233+0000 7fd1c67fc700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 0x7fd1b003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:33:59.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.234+0000 7fd1c67fc700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 0x7fd1b003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006fd0 tx=0x7fd1bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:33:59.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.249+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1b8013070 con 0x7fd1c8106a60 2024-03-23T23:33:59.454 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.454+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi149:172.21.15.149=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1c806a580 con 0x7fd1b0038fc0 2024-03-23T23:33:59.469 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.469+0000 7fd1affff700 1 -- 172.21.15.174:0/1905027253 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd1c806a580 con 0x7fd1b0038fc0 2024-03-23T23:33:59.469 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 msgr2=0x7fd1b003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 0x7fd1b003b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc006fd0 tx=0x7fd1bc006e40 comp rx=0 tx=0).stop 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 msgr2=0x7fd1c81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005850 tx=0x7fd1b8005030 comp rx=0 tx=0).stop 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 shutdown_connections 2024-03-23T23:33:59.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd1b0038fc0 0x7fd1b003b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:59.472 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 --2- 172.21.15.174:0/1905027253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1c8106a60 0x7fd1c81984f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:33:59.472 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 >> 172.21.15.174:0/1905027253 conn(0x7fd1c80fc550 msgr2=0x7fd1c8108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:33:59.472 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 shutdown_connections 2024-03-23T23:33:59.472 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:33:59.471+0000 7fd1cd333700 1 -- 172.21.15.174:0/1905027253 wait complete. 2024-03-23T23:34:00.049 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.b.service 2024-03-23T23:34:00.052 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.c.service 2024-03-23T23:34:00.054 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T23:34:00.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:00.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:00.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:33:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:00 smithi099 ceph-mon[31715]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi149:172.21.15.149=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:00 smithi099 ceph-mon[31715]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi149:172.21.15.149=b;smithi174:172.21.15.174=c;count:3 2024-03-23T23:34:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:00 smithi099 ceph-mon[31715]: Deploying daemon mon.c on smithi174 2024-03-23T23:34:02.074 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:34:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:01 smithi099 ceph-mon[31715]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.884+0000 7fa22865e700 1 -- 172.21.15.174:0/1447740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 msgr2=0x7fa2201022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.884+0000 7fa22865e700 1 --2- 172.21.15.174:0/1447740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa2201022e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b0a0 tx=0x7fa21c00b3b0 comp rx=0 tx=0).stop 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- 172.21.15.174:0/1447740093 shutdown_connections 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 --2- 172.21.15.174:0/1447740093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa2201022e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- 172.21.15.174:0/1447740093 >> 172.21.15.174:0/1447740093 conn(0x7fa2200fd790 msgr2=0x7fa2200ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:03.886 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- 172.21.15.174:0/1447740093 shutdown_connections 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- 172.21.15.174:0/1447740093 wait complete. 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 Processor -- start 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- start start 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.885+0000 7fa22865e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22010c2d0 con 0x7fa220101f00 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.886+0000 7fa2263fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.886+0000 7fa2263fa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.174:60714/0 (socket says 172.21.15.174:60714) 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.886+0000 7fa2263fa700 1 -- 172.21.15.174:0/374973757 learned_addr learned my addr 172.21.15.174:0/374973757 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:34:03.887 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.886+0000 7fa2263fa700 1 -- 172.21.15.174:0/374973757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220197c40 con 0x7fa220101f00 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa2263fa700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa21c015040 tx=0x7fa21c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa21c017040 con 0x7fa220101f00 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa220197ed0 con 0x7fa220101f00 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22019bfd0 con 0x7fa220101f00 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa21c022070 con 0x7fa220101f00 2024-03-23T23:34:03.888 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.887+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa21c004190 con 0x7fa220101f00 2024-03-23T23:34:03.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.888+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa204005330 con 0x7fa220101f00 2024-03-23T23:34:03.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.889+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa21c00bc10 con 0x7fa220101f00 2024-03-23T23:34:03.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.889+0000 7fa2177fe700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 0x7fa20c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:03.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.889+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa21c00f050 con 0x7fa220101f00 2024-03-23T23:34:03.889 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.889+0000 7fa225bf9700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 0x7fa20c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:03.890 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.890+0000 7fa225bf9700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 0x7fa20c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa210006fd0 tx=0x7fa210006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:03.904 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:03.904+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa21c007e20 con 0x7fa220101f00 2024-03-23T23:34:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:03 smithi099 ceph-mon[31715]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:04.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.205+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa2040058f0 con 0x7fa220101f00 2024-03-23T23:34:04.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.206+0000 7fa2177fe700 1 -- 172.21.15.174:0/374973757 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fa21c01ce50 con 0x7fa220101f00 2024-03-23T23:34:04.206 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:34:04.206 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","modified":"2024-03-23T23:31:50.566241Z","created":"2024-03-23T23:31:50.566241Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T23:34:04.208 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.208+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 msgr2=0x7fa20c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:04.208 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.208+0000 7fa22865e700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 0x7fa20c03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa210006fd0 tx=0x7fa210006e40 comp rx=0 tx=0).stop 2024-03-23T23:34:04.208 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.208+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 msgr2=0x7fa220197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:04.208 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.208+0000 7fa22865e700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa21c015040 tx=0x7fa21c003d50 comp rx=0 tx=0).stop 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 shutdown_connections 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa20c038fc0 0x7fa20c03b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 --2- 172.21.15.174:0/374973757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa220101f00 0x7fa220197700 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 >> 172.21.15.174:0/374973757 conn(0x7fa2200fd790 msgr2=0x7fa2201038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 shutdown_connections 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:04.209+0000 7fa22865e700 1 -- 172.21.15.174:0/374973757 wait complete. 2024-03-23T23:34:04.209 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-23T23:34:04.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:04 smithi174 systemd[1]: Started Ceph mon.c for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:34:05.841 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T23:34:05.841 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T23:34:06.086 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: mon.a calling monitor election 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: mon.c calling monitor election 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: fsmap 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: mgrmap e13: x(active, since 62s) 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: overall HEALTH_OK 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:09.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:34:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: mon.a calling monitor election 2024-03-23T23:34:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: mon.c calling monitor election 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T23:34:09.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: fsmap 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: mgrmap e13: x(active, since 62s) 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: overall HEALTH_OK 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:09.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:10.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:10 smithi174 ceph-mon[37002]: Deploying daemon mon.b on smithi149 2024-03-23T23:34:10.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:10.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/2051599069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 msgr2=0x7fb7c0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:10.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/2051599069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7c0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb7d406a3b0 tx=0x7fb7cc009e80 comp rx=0 tx=0).stop 2024-03-23T23:34:10.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/2051599069 shutdown_connections 2024-03-23T23:34:10.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/2051599069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7c0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/2051599069 >> 172.21.15.174:0/2051599069 conn(0x7fb7d40fc660 msgr2=0x7fb7d40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/2051599069 shutdown_connections 2024-03-23T23:34:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.498+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/2051599069 wait complete. 2024-03-23T23:34:10.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 Processor -- start 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 -- start start 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7d419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4109cf0 con 0x7fb7c00035c0 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d9fb3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4109b70 con 0x7fb7d419db10 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55158/0 (socket says 172.21.15.174:55158) 2024-03-23T23:34:10.500 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 -- 172.21.15.174:0/154414636 learned_addr learned my addr 172.21.15.174:0/154414636 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 -- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 msgr2=0x7fb7d41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 -- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 msgr2=0x7fb7d41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.499+0000 7fb7d2ffd700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.500+0000 7fb7d37fe700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7d419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.500+0000 7fb7d37fe700 1 -- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 msgr2=0x7fb7d41976a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.500+0000 7fb7d37fe700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:10.501 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.500+0000 7fb7d37fe700 1 -- 172.21.15.174:0/154414636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d419e090 con 0x7fb7c00035c0 2024-03-23T23:34:10.502 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d37fe700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7d419d5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc007590 tx=0x7fb7cc017890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:10.502 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb7cc005260 con 0x7fb7c00035c0 2024-03-23T23:34:10.502 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4197c40 con 0x7fb7c00035c0 2024-03-23T23:34:10.502 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7cc024030 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d41981d0 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.501+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb7cc020a70 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.502+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d406abb0 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.503+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fb7cc01e070 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.503+0000 7fb7d0ff9700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 0x7fb7b403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.503+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb7cc01c070 con 0x7fb7c00035c0 2024-03-23T23:34:10.503 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.503+0000 7fb7d2ffd700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 0x7fb7b403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:10.504 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.504+0000 7fb7d2ffd700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 0x7fb7b403b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009180 tx=0x7fb7bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:10.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.518+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb7cc01fe40 con 0x7fb7c00035c0 2024-03-23T23:34:10.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:10 smithi099 ceph-mon[31715]: Deploying daemon mon.b on smithi149 2024-03-23T23:34:10.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:10.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.792+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb7d4198bd0 con 0x7fb7c00035c0 2024-03-23T23:34:10.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.793+0000 7fb7d0ff9700 1 -- 172.21.15.174:0/154414636 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fb7cc020440 con 0x7fb7c00035c0 2024-03-23T23:34:10.793 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:34:10.793 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","modified":"2024-03-23T23:34:04.236785Z","created":"2024-03-23T23:31:50.566241Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T23:34:10.795 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 msgr2=0x7fb7b403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:10.795 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 0x7fb7b403b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009180 tx=0x7fb7bc008040 comp rx=0 tx=0).stop 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 msgr2=0x7fb7d419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7d419d5d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc007590 tx=0x7fb7cc017890 comp rx=0 tx=0).stop 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 shutdown_connections 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7b4038d60 0x7fb7b403b220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c00035c0 0x7fb7d419d5d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 --2- 172.21.15.174:0/154414636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d419db10 0x7fb7d41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 >> 172.21.15.174:0/154414636 conn(0x7fb7d40fc660 msgr2=0x7fb7d4107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 shutdown_connections 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:10.795+0000 7fb7d9fb3700 1 -- 172.21.15.174:0/154414636 wait complete. 2024-03-23T23:34:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-03-23T23:34:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:11 smithi099 ceph-mon[31715]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:11 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/154414636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T23:34:11.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:11 smithi174 ceph-mon[37002]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:11 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/154414636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T23:34:12.400 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T23:34:12.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T23:34:12.641 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:34:13.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:13 smithi149 systemd[1]: Started Ceph mon.b for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:34:18.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 -- 172.21.15.174:0/2601353864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6040035c0 msgr2=0x7fd604005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:18.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2601353864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6040035c0 0x7fd604005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd61c100a80 tx=0x7fd61000fe60 comp rx=0 tx=0).stop 2024-03-23T23:34:18.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 -- 172.21.15.174:0/2601353864 shutdown_connections 2024-03-23T23:34:18.363 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2601353864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6040035c0 0x7fd604005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 -- 172.21.15.174:0/2601353864 >> 172.21.15.174:0/2601353864 conn(0x7fd61c0fc640 msgr2=0x7fd61c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:18.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 -- 172.21.15.174:0/2601353864 shutdown_connections 2024-03-23T23:34:18.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.363+0000 7fd6224f3700 1 -- 172.21.15.174:0/2601353864 wait complete. 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.364+0000 7fd6224f3700 1 Processor -- start 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.364+0000 7fd6224f3700 1 -- start start 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 0x7fd61c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c108b90 con 0x7fd61c199ff0 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c108a10 con 0x7fd61c19ec40 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd6224f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c108d10 con 0x7fd6040035c0 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50918/0 (socket says 172.21.15.174:50918) 2024-03-23T23:34:18.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 -- 172.21.15.174:0/2661496971 learned_addr learned my addr 172.21.15.174:0/2661496971 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61b7fe700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 msgr2=0x7fd61c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 msgr2=0x7fd61c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61bfff700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd620a90700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 0x7fd61c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61b7fe700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 msgr2=0x7fd61c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61b7fe700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 msgr2=0x7fd61c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61b7fe700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T23:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.365+0000 7fd61b7fe700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T23:34:18.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 msgr2=0x7fd61c19e700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:34:18.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 msgr2=0x7fd61c199ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:34:18.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 -- 172.21.15.174:0/2661496971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c1a6680 con 0x7fd61c199ff0 2024-03-23T23:34:18.369 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.366+0000 7fd620a90700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 0x7fd61c19a4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd61400bd50 tx=0x7fd61400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:18.373 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.373+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614019070 con 0x7fd61c199ff0 2024-03-23T23:34:18.374 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.374+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd61400cd10 con 0x7fd61c199ff0 2024-03-23T23:34:18.374 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.374+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140127e0 con 0x7fd61c199ff0 2024-03-23T23:34:18.374 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.374+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c1a6910 con 0x7fd61c199ff0 2024-03-23T23:34:18.374 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.374+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c1a6dd0 con 0x7fd61c199ff0 2024-03-23T23:34:18.375 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.375+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c0637c0 con 0x7fd61c199ff0 2024-03-23T23:34:18.375 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.375+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd61401c070 con 0x7fd61c199ff0 2024-03-23T23:34:18.375 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.376+0000 7fd6197fa700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 0x7fd60803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:18.376 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.376+0000 7fd61bfff700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 0x7fd60803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:18.377 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.377+0000 7fd61bfff700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 0x7fd60803b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd610001eb0 tx=0x7fd610005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:18.379 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.379+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd614014440 con 0x7fd61c199ff0 2024-03-23T23:34:18.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.395+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd614017cb0 con 0x7fd61c199ff0 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: mon.c calling monitor election 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: mon.a calling monitor election 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: fsmap 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: mgrmap e13: x(active, since 71s) 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-23T23:34:18.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:18.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:18.683 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.683+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd61c1a7130 con 0x7fd61c199ff0 2024-03-23T23:34:18.684 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.684+0000 7fd6197fa700 1 -- 172.21.15.174:0/2661496971 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fd61401dc30 con 0x7fd61c199ff0 2024-03-23T23:34:18.684 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-23T23:34:18.684 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","modified":"2024-03-23T23:34:13.289728Z","created":"2024-03-23T23:31:50.566241Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T23:34:18.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.686+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 msgr2=0x7fd60803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:18.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.686+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 0x7fd60803b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd610001eb0 tx=0x7fd610005de0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.686+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 msgr2=0x7fd61c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.686+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 0x7fd61c19a4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd61400bd50 tx=0x7fd61400be30 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 shutdown_connections 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd608039450 0x7fd60803b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c199ff0 0x7fd61c19a4a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6040035c0 0x7fd61c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 --2- 172.21.15.174:0/2661496971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c19ec40 0x7fd61c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 >> 172.21.15.174:0/2661496971 conn(0x7fd61c0fc640 msgr2=0x7fd61c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 shutdown_connections 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:18.687+0000 7fd6224f3700 1 -- 172.21.15.174:0/2661496971 wait complete. 2024-03-23T23:34:18.687 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-03-23T23:34:18.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: mon.c calling monitor election 2024-03-23T23:34:18.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:34:18.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: mon.a calling monitor election 2024-03-23T23:34:18.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: fsmap 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: mgrmap e13: x(active, since 71s) 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-23T23:34:18.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-23T23:34:18.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-23T23:34:18.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:18.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:19.244 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-23T23:34:19.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/2661496971' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T23:34:19.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:19.653 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/2661496971' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T23:34:19.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: mon.c calling monitor election 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: mon.a calling monitor election 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: fsmap 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: mgrmap e13: x(active, since 71s) 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-23T23:34:20.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/2661496971' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T23:34:20.646 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mon.b calling monitor election 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mon.b calling monitor election 2024-03-23T23:34:21.037 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mon.c calling monitor election 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mon.a calling monitor election 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: fsmap 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: mgrmap e13: x(active, since 73s) 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: overall HEALTH_OK 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:21 smithi099 ceph-mon[31715]: Updating smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mon.b calling monitor election 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mon.b calling monitor election 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mon.c calling monitor election 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mon.a calling monitor election 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: fsmap 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: mgrmap e13: x(active, since 73s) 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: overall HEALTH_OK 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:21 smithi149 ceph-mon[36083]: Updating smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mon.b calling monitor election 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mon.b calling monitor election 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mon.c calling monitor election 2024-03-23T23:34:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mon.a calling monitor election 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: fsmap 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: mgrmap e13: x(active, since 73s) 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: overall HEALTH_OK 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:21.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:21 smithi174 ceph-mon[37002]: Updating smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 -- 172.21.15.99:0/4249817201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d0100910 msgr2=0x7f41d0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 --2- 172.21.15.99:0/4249817201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d0100910 0x7f41d0100cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009910 tx=0x7f41c0009c20 comp rx=0 tx=0).stop 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 -- 172.21.15.99:0/4249817201 shutdown_connections 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 --2- 172.21.15.99:0/4249817201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d0100910 0x7f41d0100cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 -- 172.21.15.99:0/4249817201 >> 172.21.15.99:0/4249817201 conn(0x7f41d00776d0 msgr2=0x7f41d00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 -- 172.21.15.99:0/4249817201 shutdown_connections 2024-03-23T23:34:21.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.992+0000 7f41d6600700 1 -- 172.21.15.99:0/4249817201 wait complete. 2024-03-23T23:34:21.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.993+0000 7f41d6600700 1 Processor -- start 2024-03-23T23:34:21.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.993+0000 7f41d6600700 1 -- start start 2024-03-23T23:34:21.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 0x7f41d019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:21.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41d0199ea0 0x7f41d019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d010e3a0 con 0x7f41d019fce0 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d010e220 con 0x7f41d0100910 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41d6600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d010e520 con 0x7f41d0199ea0 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41cea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41cea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55260/0 (socket says 172.21.15.99:55260) 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41cea00700 1 -- 172.21.15.99:0/257570782 learned_addr learned my addr 172.21.15.99:0/257570782 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41cf400700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 0x7f41d019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:21.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.994+0000 7f41cfe00700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41d0199ea0 0x7f41d019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 -- 172.21.15.99:0/257570782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41d0199ea0 msgr2=0x7f41d019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41d0199ea0 0x7f41d019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 -- 172.21.15.99:0/257570782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 msgr2=0x7f41d019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 0x7f41d019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 -- 172.21.15.99:0/257570782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d01a6670 con 0x7f41d019fce0 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cf400700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 0x7f41d019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41cea00700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b780 tx=0x7f41bc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:21.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc016070 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41bc0036a0 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.995+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d01a68a0 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.996+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc011af0 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.996+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d01a6e30 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.997+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f41bc019070 con 0x7f41d019fce0 2024-03-23T23:34:21.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.997+0000 7f41c7e00700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 0x7f41b403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:21.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.997+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f41bc04aae0 con 0x7f41d019fce0 2024-03-23T23:34:21.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.998+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d00637c0 con 0x7f41d019fce0 2024-03-23T23:34:21.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.998+0000 7f41cf400700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 0x7f41b403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:21.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:21.998+0000 7f41cf400700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 0x7f41b403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009750 tx=0x7f41c000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:22.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.003+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41bc0199b0 con 0x7f41d019fce0 2024-03-23T23:34:22.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.235+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f41d01a7060 con 0x7f41d019fce0 2024-03-23T23:34:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.236+0000 7f41c7e00700 1 -- 172.21.15.99:0/257570782 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f41bc019c10 con 0x7f41d019fce0 2024-03-23T23:34:22.236 INFO:teuthology.orchestra.run.smithi099.stdout:# minimal ceph.conf for 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:34:22.236 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-03-23T23:34:22.236 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:34:22.236 INFO:teuthology.orchestra.run.smithi099.stdout: mon_host = [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.238+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 msgr2=0x7f41b403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.238+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 0x7f41b403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009750 tx=0x7f41c000b540 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.238+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 msgr2=0x7f41d0199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.238+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b780 tx=0x7f41bc00ba90 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 shutdown_connections 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f41b4039450 0x7f41b403b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41d0199ea0 0x7f41d019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41d0100910 0x7f41d019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 --2- 172.21.15.99:0/257570782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41d019fce0 0x7f41d0199960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 >> 172.21.15.99:0/257570782 conn(0x7f41d00776d0 msgr2=0x7f41d0108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 shutdown_connections 2024-03-23T23:34:22.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:22.239+0000 7f41d6600700 1 -- 172.21.15.99:0/257570782 wait complete. 2024-03-23T23:34:22.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:22.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: Reconfiguring daemon mon.a on smithi099 2024-03-23T23:34:22.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:22 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:22.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: Reconfiguring daemon mon.a on smithi099 2024-03-23T23:34:22.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:22 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T23:34:22.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:22.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:22.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:22.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: Reconfiguring daemon mon.a on smithi099 2024-03-23T23:34:22.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:22 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:22.962 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-23T23:34:22.963 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:34:22.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T23:34:23.046 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:34:23.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:34:23.164 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:34:23.164 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T23:34:23.197 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:34:23.197 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:34:23.267 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:34:23.267 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T23:34:23.296 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:23 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/257570782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:23.297 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:23 smithi174 ceph-mon[37002]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:23.301 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:34:23.302 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T23:34:23.337 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:23 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/257570782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:23.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:23 smithi099 ceph-mon[31715]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:23.372 INFO:tasks.cephadm:Adding mgr.x on smithi099 2024-03-23T23:34:23.372 INFO:tasks.cephadm:Adding mgr.y on smithi149 2024-03-23T23:34:23.372 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-03-23T23:34:23.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi099=x;smithi149=y;smithi174=z' 2024-03-23T23:34:23.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:23 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/257570782' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:23.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:23 smithi149 ceph-mon[36083]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:23.646 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:34:25.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 -- 172.21.15.174:0/2391104500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 msgr2=0x7f7460103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:25.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 --2- 172.21.15.174:0/2391104500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460103800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7454009910 tx=0x7f7454009c20 comp rx=0 tx=0).stop 2024-03-23T23:34:25.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 -- 172.21.15.174:0/2391104500 shutdown_connections 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 --2- 172.21.15.174:0/2391104500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460103800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 -- 172.21.15.174:0/2391104500 >> 172.21.15.174:0/2391104500 conn(0x7f74600fd280 msgr2=0x7f74600ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 -- 172.21.15.174:0/2391104500 shutdown_connections 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 -- 172.21.15.174:0/2391104500 wait complete. 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.092+0000 7f7466b8c700 1 Processor -- start 2024-03-23T23:34:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 -- start start 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7460197ff0 0x7f746019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460107e50 con 0x7f74601033e0 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460107cd0 con 0x7f7460197ff0 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7466b8c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460107fd0 con 0x7f7460197b70 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:36606/0 (socket says 172.21.15.174:36606) 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 -- 172.21.15.174:0/1220716907 learned_addr learned my addr 172.21.15.174:0/1220716907 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7465129700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7460197ff0 0x7f746019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f7464928700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 -- 172.21.15.174:0/1220716907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7460197ff0 msgr2=0x7f746019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7460197ff0 0x7f746019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 -- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 msgr2=0x7f7460197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.093+0000 7f745ffff700 1 -- 172.21.15.174:0/1220716907 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746019fa40 con 0x7f7460197b70 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f745ffff700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f744c00b780 tx=0x7f744c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c004230 con 0x7f7460197b70 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f744c0043d0 con 0x7f7460197b70 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c011c90 con 0x7f7460197b70 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746019fd30 con 0x7f7460197b70 2024-03-23T23:34:25.100 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.094+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74601a0290 con 0x7f7460197b70 2024-03-23T23:34:25.101 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.095+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f744c01a400 con 0x7f7460197b70 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.095+0000 7f745dffb700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 0x7f745003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.095+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f744c04bb90 con 0x7f7460197b70 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.095+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74600637c0 con 0x7f7460197b70 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.095+0000 7f7464928700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 0x7f745003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.096+0000 7f7464928700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 0x7f745003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f745400b5c0 tx=0x7f745401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:25.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.100+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f744c018750 con 0x7f7460197b70 2024-03-23T23:34:25.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.310+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi149=y;smithi174=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f74600625d0 con 0x7f7450039450 2024-03-23T23:34:25.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.323+0000 7f745dffb700 1 -- 172.21.15.174:0/1220716907 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f74600625d0 con 0x7f7450039450 2024-03-23T23:34:25.323 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-03-23T23:34:25.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 msgr2=0x7f745003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:25.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 0x7f745003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f745400b5c0 tx=0x7f745401a040 comp rx=0 tx=0).stop 2024-03-23T23:34:25.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 msgr2=0x7f746019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:25.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f744c00b780 tx=0x7f744c00ba90 comp rx=0 tx=0).stop 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 shutdown_connections 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7450039450 0x7f745003b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7460197ff0 0x7f746019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74601033e0 0x7f7460197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 --2- 172.21.15.174:0/1220716907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7460197b70 0x7f746019cbe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 >> 172.21.15.174:0/1220716907 conn(0x7f74600fd280 msgr2=0x7f74600fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 shutdown_connections 2024-03-23T23:34:25.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:34:25.325+0000 7f7466b8c700 1 -- 172.21.15.174:0/1220716907 wait complete. 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: Reconfiguring mon.b (monmap changed)... 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:25.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:25.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: Reconfiguring daemon mon.b on smithi149 2024-03-23T23:34:25.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.917 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.y.service 2024-03-23T23:34:25.919 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.z.service 2024-03-23T23:34:25.921 INFO:tasks.cephadm:Deploying OSDs... 2024-03-23T23:34:25.922 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:34:25.922 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T23:34:25.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:25.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: Reconfiguring mon.b (monmap changed)... 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: Reconfiguring daemon mon.b on smithi149 2024-03-23T23:34:25.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:25.978 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T23:34:25.978 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-23 23:33:20.653589543 +0000 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-23 23:28:06.267224541 +0000 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-23 23:28:06.267224541 +0000 2024-03-23T23:34:26.030 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-23 23:28:06.267224541 +0000 2024-03-23T23:34:26.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: Reconfiguring mon.b (monmap changed)... 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:26.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: Reconfiguring daemon mon.b on smithi149 2024-03-23T23:34:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.122 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-23T23:34:26.122 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-23T23:34:26.122 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000380758 s, 1.3 MB/s 2024-03-23T23:34:26.127 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T23:34:26.184 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-23 23:33:20.725587850 +0000 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-23 23:28:06.812211583 +0000 2024-03-23T23:34:26.233 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-23 23:28:06.812211583 +0000 2024-03-23T23:34:26.234 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-23 23:28:06.812211583 +0000 2024-03-23T23:34:26.234 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T23:34:26.326 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-23T23:34:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-23T23:34:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000364232 s, 1.4 MB/s 2024-03-23T23:34:26.331 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T23:34:26.382 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-03-23T23:34:26.430 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T23:34:26.430 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:26.430 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-23T23:34:26.430 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:26.430 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:26.431 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-23 23:33:20.793586251 +0000 2024-03-23T23:34:26.431 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-23 23:28:07.413197293 +0000 2024-03-23T23:34:26.431 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-23 23:28:07.413197293 +0000 2024-03-23T23:34:26.431 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-23 23:28:07.413197293 +0000 2024-03-23T23:34:26.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T23:34:26.515 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-23T23:34:26.515 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-23T23:34:26.515 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000357674 s, 1.4 MB/s 2024-03-23T23:34:26.519 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T23:34:26.577 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-03-23T23:34:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T23:34:26.627 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:26.627 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-23 23:33:20.868584488 +0000 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-23 23:28:07.986183668 +0000 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-23 23:28:07.986183668 +0000 2024-03-23T23:34:26.628 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-23 23:28:07.986183668 +0000 2024-03-23T23:34:26.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T23:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-23T23:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-23T23:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.0004889 s, 1.0 MB/s 2024-03-23T23:34:26.723 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T23:34:26.777 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-23T23:34:26.777 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T23:34:26.794 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T23:34:26.794 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:26.850 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-23 23:34:12.775470140 +0000 2024-03-23T23:34:26.851 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-23 23:27:05.664711074 +0000 2024-03-23T23:34:26.851 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-23 23:27:05.664711074 +0000 2024-03-23T23:34:26.851 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-23 23:27:05.664711074 +0000 2024-03-23T23:34:26.851 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi149=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: Saving service mgr spec with placement smithi099=x;smithi149=y;smithi174=z;count:3 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:26.916 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-23T23:34:26.917 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-23T23:34:26.917 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000221528 s, 2.3 MB/s 2024-03-23T23:34:26.918 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi149=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: Saving service mgr spec with placement smithi099=x;smithi149=y;smithi174=z;count:3 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:26.934 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:26.974 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-23 23:34:12.775470140 +0000 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-23 23:27:06.002704181 +0000 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-23 23:27:06.002704181 +0000 2024-03-23T23:34:27.029 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-23 23:27:06.002704181 +0000 2024-03-23T23:34:27.030 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi149=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: Saving service mgr spec with placement smithi099=x;smithi149=y;smithi174=z;count:3 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T23:34:27.093 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:27.094 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-23T23:34:27.095 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-23T23:34:27.095 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000210088 s, 2.4 MB/s 2024-03-23T23:34:27.096 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T23:34:27.152 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-23 23:34:12.776470119 +0000 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-23 23:27:06.346697169 +0000 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-23 23:27:06.346697169 +0000 2024-03-23T23:34:27.207 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-23 23:27:06.346697169 +0000 2024-03-23T23:34:27.207 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T23:34:27.272 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-23T23:34:27.273 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-23T23:34:27.273 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000218105 s, 2.3 MB/s 2024-03-23T23:34:27.274 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T23:34:27.330 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-03-23T23:34:27.385 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T23:34:27.385 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:27.385 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-23T23:34:27.385 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:27.386 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:27.386 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-23 23:34:12.776470119 +0000 2024-03-23T23:34:27.386 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-23 23:27:06.694690075 +0000 2024-03-23T23:34:27.386 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-23 23:27:06.694690075 +0000 2024-03-23T23:34:27.386 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-23 23:27:06.694690075 +0000 2024-03-23T23:34:27.386 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T23:34:27.451 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-23T23:34:27.451 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-23T23:34:27.451 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000211287 s, 2.4 MB/s 2024-03-23T23:34:27.453 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T23:34:27.508 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-23T23:34:27.508 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T23:34:27.524 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T23:34:27.525 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-23 23:34:03.050653532 +0000 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-23 23:27:05.664742818 +0000 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-23 23:27:05.664742818 +0000 2024-03-23T23:34:27.580 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-23 23:27:05.664742818 +0000 2024-03-23T23:34:27.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T23:34:27.647 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-23T23:34:27.647 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-23T23:34:27.647 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000188688 s, 2.7 MB/s 2024-03-23T23:34:27.648 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T23:34:27.704 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-03-23T23:34:27.759 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T23:34:27.759 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:27.759 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-23T23:34:27.759 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:27.760 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:27.760 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-23 23:34:03.050653532 +0000 2024-03-23T23:34:27.760 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-23 23:27:06.013737930 +0000 2024-03-23T23:34:27.760 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-23 23:27:06.013737930 +0000 2024-03-23T23:34:27.760 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-23 23:27:06.013737930 +0000 2024-03-23T23:34:27.760 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T23:34:27.824 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-23T23:34:27.825 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-23T23:34:27.825 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000203956 s, 2.5 MB/s 2024-03-23T23:34:27.826 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T23:34:27.882 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:27 smithi149 ceph-mon[36083]: Reconfiguring mon.c (monmap changed)... 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:27 smithi149 ceph-mon[36083]: Reconfiguring daemon mon.c on smithi174 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:27 smithi149 ceph-mon[36083]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:27 smithi174 ceph-mon[37002]: Reconfiguring mon.c (monmap changed)... 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:27 smithi174 ceph-mon[37002]: Reconfiguring daemon mon.c on smithi174 2024-03-23T23:34:27.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:27 smithi174 ceph-mon[37002]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:28.027 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T23:34:28.027 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:28.027 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-23T23:34:28.027 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:28.027 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:28.028 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-23 23:34:03.050653532 +0000 2024-03-23T23:34:28.028 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-23 23:27:06.347733255 +0000 2024-03-23T23:34:28.028 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-23 23:27:06.347733255 +0000 2024-03-23T23:34:28.028 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-23 23:27:06.347733255 +0000 2024-03-23T23:34:28.028 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T23:34:28.115 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-23T23:34:28.115 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-23T23:34:28.115 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000185713 s, 2.8 MB/s 2024-03-23T23:34:28.116 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T23:34:28.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:27 smithi099 ceph-mon[31715]: Reconfiguring mon.c (monmap changed)... 2024-03-23T23:34:28.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:27 smithi099 ceph-mon[31715]: Reconfiguring daemon mon.c on smithi174 2024-03-23T23:34:28.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:27 smithi099 ceph-mon[31715]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:28.182 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-03-23T23:34:28.242 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T23:34:28.242 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T23:34:28.242 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-23T23:34:28.242 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:34:28.242 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T23:34:28.243 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-23 23:34:03.051653518 +0000 2024-03-23T23:34:28.243 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-23 23:27:06.664728817 +0000 2024-03-23T23:34:28.243 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-23 23:27:06.664728817 +0000 2024-03-23T23:34:28.243 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-23 23:27:06.664728817 +0000 2024-03-23T23:34:28.243 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T23:34:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-23T23:34:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-23T23:34:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000189958 s, 2.7 MB/s 2024-03-23T23:34:28.317 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T23:34:28.375 INFO:tasks.cephadm:Deploying osd.0 on smithi099 with /dev/vg_nvme/lv_4... 2024-03-23T23:34:28.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T23:34:28.811 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:29.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:30.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:29 smithi174 ceph-mon[37002]: Deploying daemon mgr.z on smithi174 2024-03-23T23:34:30.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:29 smithi174 ceph-mon[37002]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:30.098 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:29 smithi174 systemd[1]: Starting Ceph mgr.z for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:34:30.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:29 smithi149 ceph-mon[36083]: Deploying daemon mgr.z on smithi174 2024-03-23T23:34:30.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:29 smithi149 ceph-mon[36083]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:29 smithi099 ceph-mon[31715]: Deploying daemon mgr.z on smithi174 2024-03-23T23:34:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:29 smithi099 ceph-mon[31715]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:30.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 podman[38863]: 2024-03-23 23:34:30.098254373 +0000 UTC m=+0.105195163 container create a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True) 2024-03-23T23:34:30.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 podman[38863]: 2024-03-23 23:34:30.01048675 +0000 UTC m=+0.017427528 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:34:30.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 podman[38863]: 2024-03-23 23:34:30.284371493 +0000 UTC m=+0.291312269 container init a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-23T23:34:30.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 podman[38863]: 2024-03-23 23:34:30.287029829 +0000 UTC m=+0.293970601 container start a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:34:30.383 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 ceph-mgr[38883]: -- 172.21.15.174:0/207835509 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fe130e5a0 con 0x558fe12e3000 2024-03-23T23:34:30.384 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 bash[38863]: a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 2024-03-23T23:34:30.673 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 systemd[1]: Started Ceph mgr.z for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:34:31.172 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:30.835+0000 7f805c92a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T23:34:31.173 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:30.924+0000 7f805c92a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:31.584+0000 7f805c92a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:31.623 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:31.624 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:31.624 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: Deploying daemon mgr.y on smithi149 2024-03-23T23:34:31.624 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-mon[37002]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: Deploying daemon mgr.y on smithi149 2024-03-23T23:34:31.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:31 smithi099 ceph-mon[31715]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: Deploying daemon mgr.y on smithi149 2024-03-23T23:34:31.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:31 smithi149 ceph-mon[36083]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:31.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:31.682+0000 7f805c92a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T23:34:31.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:31.831+0000 7f805c92a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354159 s, 296 MB/s 2024-03-23T23:34:31.993 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-23T23:34:32.283 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:31.929+0000 7f805c92a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T23:34:32.284 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.014+0000 7f805c92a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T23:34:32.284 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.283+0000 7f805c92a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T23:34:32.577 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.396+0000 7f805c92a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T23:34:32.578 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.483+0000 7f805c92a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T23:34:32.790 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:32 smithi149 systemd[1]: Starting Ceph mgr.y for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:34:32.856 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-03-23T23:34:32.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.580+0000 7f805c92a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T23:34:32.923 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:32.756+0000 7f805c92a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T23:34:33.051 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:32 smithi149 podman[37451]: 2024-03-23 23:34:32.791224166 +0000 UTC m=+0.103167437 container create 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-23T23:34:33.051 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:32 smithi149 podman[37451]: 2024-03-23 23:34:32.706380741 +0000 UTC m=+0.018323976 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:34:33.051 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:32 smithi149 podman[37451]: 2024-03-23 23:34:32.976829258 +0000 UTC m=+0.288772509 container init 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-23T23:34:33.051 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:32 smithi149 podman[37451]: 2024-03-23 23:34:32.979359552 +0000 UTC m=+0.291302788 container start 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:34:33.052 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mgr[37471]: -- 172.21.15.149:0/2722912053 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ae386a5a0 con 0x558ae383e800 2024-03-23T23:34:33.052 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:33 smithi149 bash[37451]: 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 2024-03-23T23:34:33.282 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:34:33.394 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:33 smithi149 systemd[1]: Started Ceph mgr.y for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:34:33.673 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:33.673 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.673 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.893 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:33.540+0000 7fd0de591200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:33.636+0000 7fd0de591200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:33.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:33.743+0000 7f805c92a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T23:34:33.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:33.881+0000 7f805c92a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:34.545 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:34.254+0000 7f805c92a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T23:34:34.545 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:34.349+0000 7f805c92a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T23:34:34.545 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:34.545+0000 7f805c92a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:34:34.570 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:34.326+0000 7fd0de591200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T23:34:34.570 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:34.423+0000 7fd0de591200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T23:34:34.893 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:34.569+0000 7fd0de591200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T23:34:34.894 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:34.670+0000 7fd0de591200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T23:34:34.894 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:34.762+0000 7fd0de591200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T23:34:34.922 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:34.741+0000 7f805c92a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T23:34:35.276 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:35.019+0000 7f805c92a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:34:35.277 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:35.276+0000 7f805c92a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T23:34:35.315 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:35.029+0000 7fd0de591200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T23:34:35.315 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:35.138+0000 7fd0de591200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T23:34:35.315 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:35.222+0000 7fd0de591200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T23:34:35.315 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:35.315+0000 7fd0de591200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T23:34:35.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 -- 172.21.15.99:0/1649691983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a24074b90 msgr2=0x7f6a24075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 --2- 172.21.15.99:0/1649691983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a24074b90 0x7f6a24075010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a100098f0 tx=0x7f6a10009c00 comp rx=0 tx=0).stop 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 -- 172.21.15.99:0/1649691983 shutdown_connections 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 --2- 172.21.15.99:0/1649691983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a2410a0f0 0x7f6a2410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 --2- 172.21.15.99:0/1649691983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 0x7f6a240745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 --2- 172.21.15.99:0/1649691983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a24074b90 0x7f6a24075010 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 -- 172.21.15.99:0/1649691983 >> 172.21.15.99:0/1649691983 conn(0x7f6a240fd7b0 msgr2=0x7f6a240ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 -- 172.21.15.99:0/1649691983 shutdown_connections 2024-03-23T23:34:35.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.406+0000 7f6a2a600700 1 -- 172.21.15.99:0/1649691983 wait complete. 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.407+0000 7f6a2a600700 1 Processor -- start 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.407+0000 7f6a2a600700 1 -- start start 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 0x7f6a2419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a2410a0f0 0x7f6a241a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24110020 con 0x7f6a2410a0f0 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2410fea0 con 0x7f6a24076160 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a2a600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241101a0 con 0x7f6a24074b90 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57398/0 (socket says 172.21.15.99:57398) 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 -- 172.21.15.99:0/151316666 learned_addr learned my addr 172.21.15.99:0/151316666 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a22a00700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 0x7f6a2419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:35.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23e00700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a2410a0f0 0x7f6a241a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 -- 172.21.15.99:0/151316666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 msgr2=0x7f6a2419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 0x7f6a2419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 -- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a2410a0f0 msgr2=0x7f6a241a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.408+0000 7f6a23400700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a2410a0f0 0x7f6a241a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a23400700 1 -- 172.21.15.99:0/151316666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a241a2270 con 0x7f6a24074b90 2024-03-23T23:34:35.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a23400700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400eb60 tx=0x7f6a1400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:35.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400cc90 con 0x7f6a24074b90 2024-03-23T23:34:35.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a14004500 con 0x7f6a24074b90 2024-03-23T23:34:35.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a241a2560 con 0x7f6a24074b90 2024-03-23T23:34:35.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14010450 con 0x7f6a24074b90 2024-03-23T23:34:35.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.409+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241a2af0 con 0x7f6a24074b90 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.411+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24068e50 con 0x7f6a24074b90 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.411+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f6a140105f0 con 0x7f6a24074b90 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.411+0000 7f6a1be00700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 0x7f6a0803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.411+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a1404c3b0 con 0x7f6a24074b90 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.412+0000 7f6a22a00700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 0x7f6a0803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:34:35.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.412+0000 7f6a22a00700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 0x7f6a0803b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000fb70 tx=0x7f6a10019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:34:35.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.417+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a1401fe20 con 0x7f6a24074b90 2024-03-23T23:34:35.644 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:35.494+0000 7fd0de591200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T23:34:35.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:35.654+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a240625d0 con 0x7f6a08039200 2024-03-23T23:34:35.666 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:35.507+0000 7f805c92a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T23:34:35.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:35.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:35.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:35.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:35.925 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:35.925 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:34:35.925 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:35 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:34:36.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:36.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.042 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:34:36.043 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:35 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:34:36.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:34:36.147 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:35 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:34:36.668 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:36.288+0000 7f805c92a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T23:34:36.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-mon[36083]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T23:34:36.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-mon[36083]: Reconfiguring daemon mgr.x on smithi099 2024-03-23T23:34:36.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-mon[36083]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:36.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.895 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:36.462+0000 7fd0de591200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T23:34:36.895 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:36.588+0000 7fd0de591200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T23:34:36.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-mon[37002]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T23:34:36.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-mon[37002]: Reconfiguring daemon mgr.x on smithi099 2024-03-23T23:34:36.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-mon[37002]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:36.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:36.923 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:36.850+0000 7f805c92a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T23:34:36.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:36 smithi099 ceph-mon[31715]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T23:34:36.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:36 smithi099 ceph-mon[31715]: Reconfiguring daemon mgr.x on smithi099 2024-03-23T23:34:36.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:36 smithi099 ceph-mon[31715]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:34:36.981 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:36 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:37.208 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:36.926+0000 7fd0de591200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T23:34:37.208 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:37.013+0000 7fd0de591200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T23:34:37.208 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:37.207+0000 7fd0de591200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:34:37.423 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:36.945+0000 7f805c92a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T23:34:37.423 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z[38879]: 2024-03-23T23:34:37.154+0000 7f805c92a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T23:34:37.644 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:37.391+0000 7fd0de591200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T23:34:37.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:37.685+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7f6a14017cd0 con 0x7f6a24074b90 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: Standby manager daemon z started 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:37.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:37.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:37.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:37.677+0000 7fd0de591200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:37.944+0000 7fd0de591200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: Standby manager daemon z started 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:37.945 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:37.946 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:38.064 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: Standby manager daemon z started 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.174:0/2593906409' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:38.065 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:38.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:38.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:34:38.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:38.393 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:38.178+0000 7fd0de591200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T23:34:38.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:38 smithi099 ceph-mon[31715]: mgrmap e14: x(active, since 91s), standbys: z 2024-03-23T23:34:38.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:38 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T23:34:38.980 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:38 smithi149 ceph-mon[36083]: mgrmap e14: x(active, since 91s), standbys: z 2024-03-23T23:34:38.980 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:38 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T23:34:39.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:38 smithi174 ceph-mon[37002]: mgrmap e14: x(active, since 91s), standbys: z 2024-03-23T23:34:39.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:38 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T23:34:39.393 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:38.979+0000 7fd0de591200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T23:34:39.868 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:39 smithi149 ceph-mon[36083]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:39.868 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:39.555+0000 7fd0de591200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T23:34:39.868 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:39.652+0000 7fd0de591200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T23:34:39.868 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:34:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y[37467]: 2024-03-23T23:34:39.867+0000 7fd0de591200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T23:34:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:39 smithi099 ceph-mon[31715]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:40.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:39 smithi174 ceph-mon[37002]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:40.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:34:40.715+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mon.1 v2:172.21.15.174:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f6a1405a070 con 0x7f6a24074b90 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: Standby manager daemon y started 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]: dispatch 2024-03-23T23:34:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]': finished 2024-03-23T23:34:41.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T23:34:41.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:40 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: Standby manager daemon y started 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]: dispatch 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]': finished 2024-03-23T23:34:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T23:34:41.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:40 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: Standby manager daemon y started 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='mgr.? 172.21.15.149:0/2251723645' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]: dispatch 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2543021971' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "819aae5f-09d9-444c-953a-ec0173dc128e"}]': finished 2024-03-23T23:34:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T23:34:41.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:40 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:34:42.060 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:41 smithi174 ceph-mon[37002]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:42.060 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:41 smithi174 ceph-mon[37002]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-03-23T23:34:42.060 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T23:34:42.060 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:41 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1431595632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:34:42.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:41 smithi099 ceph-mon[31715]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:41 smithi099 ceph-mon[31715]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-03-23T23:34:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T23:34:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:41 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1431595632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:34:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:41 smithi149 ceph-mon[36083]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:41 smithi149 ceph-mon[36083]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-03-23T23:34:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T23:34:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:41 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1431595632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:34:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:43 smithi099 ceph-mon[31715]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:44.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:43 smithi149 ceph-mon[36083]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:44.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:43 smithi174 ceph-mon[37002]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:46.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:45 smithi099 ceph-mon[31715]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:46.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:45 smithi149 ceph-mon[36083]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:46.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:45 smithi174 ceph-mon[37002]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:47.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:47 smithi099 ceph-mon[31715]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:48.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:47 smithi149 ceph-mon[36083]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:48.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:47 smithi174 ceph-mon[37002]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:49 smithi099 ceph-mon[31715]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T23:34:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:49.986 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:49 smithi174 ceph-mon[37002]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:49.987 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T23:34:49.987 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:49 smithi149 ceph-mon[36083]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T23:34:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:34:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:50 smithi099 ceph-mon[31715]: Deploying daemon osd.0 on smithi099 2024-03-23T23:34:51.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:50 smithi149 ceph-mon[36083]: Deploying daemon osd.0 on smithi099 2024-03-23T23:34:51.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:50 smithi174 ceph-mon[37002]: Deploying daemon osd.0 on smithi099 2024-03-23T23:34:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:51 smithi099 ceph-mon[31715]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:52.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:51 smithi149 ceph-mon[36083]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:51 smithi174 ceph-mon[37002]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:53.936 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:53 smithi099 ceph-mon[31715]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:54.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:53 smithi149 ceph-mon[36083]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:54.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:53 smithi174 ceph-mon[37002]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:56.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:55 smithi099 ceph-mon[31715]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:56.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:55 smithi149 ceph-mon[36083]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:55 smithi174 ceph-mon[37002]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:57 smithi099 ceph-mon[31715]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:58.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:57 smithi149 ceph-mon[36083]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:58.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:57 smithi174 ceph-mon[37002]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:34:59.080 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.080 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.080 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:59.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:59.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:34:59.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:34:59.965 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:34:59 smithi099 ceph-mon[31715]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:34:59 smithi149 ceph-mon[36083]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:00.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:34:59 smithi174 ceph-mon[37002]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:01 smithi099 ceph-mon[31715]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:01 smithi149 ceph-mon[36083]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T23:35:02.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:01 smithi174 ceph-mon[37002]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T23:35:02.705 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 0 on host 'smithi099' 2024-03-23T23:35:02.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.704+0000 7f6a1be00700 1 -- 172.21.15.99:0/151316666 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a240625d0 con 0x7f6a08039200 2024-03-23T23:35:02.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 msgr2=0x7f6a0803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:02.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 0x7f6a0803b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000fb70 tx=0x7f6a10019040 comp rx=0 tx=0).stop 2024-03-23T23:35:02.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 msgr2=0x7f6a2419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:02.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400eb60 tx=0x7f6a1400ee70 comp rx=0 tx=0).stop 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 shutdown_connections 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6a08039200 0x7f6a0803b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a2410a0f0 0x7f6a241a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a24074b90 0x7f6a2419a660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 --2- 172.21.15.99:0/151316666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a24076160 0x7f6a2419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.708+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 >> 172.21.15.99:0/151316666 conn(0x7f6a240fd7b0 msgr2=0x7f6a24106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.709+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 shutdown_connections 2024-03-23T23:35:02.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:02.709+0000 7f6a2a600700 1 -- 172.21.15.99:0/151316666 wait complete. 2024-03-23T23:35:03.488 DEBUG:teuthology.orchestra.run.smithi099:osd.0> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.0.service 2024-03-23T23:35:03.493 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2024-03-23T23:35:03.493 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.685 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:03.739 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:35:03.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T23:35:03.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T23:35:03.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:35:03.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:03.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:03.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:03.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:04.362 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0[44273]: 2024-03-23T23:35:04.040+0000 7f75b3800700 -1 osd.0 0 waiting for initial osdmap 2024-03-23T23:35:04.362 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0[44273]: 2024-03-23T23:35:04.055+0000 7f75a9600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:35:04.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:04 smithi174 ceph-mon[37002]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:35:04.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:04 smithi174 ceph-mon[37002]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T23:35:04.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:04 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:04 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:04 smithi174 ceph-mon[37002]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' 2024-03-23T23:35:04.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-mon[31715]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:35:04.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-mon[31715]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T23:35:04.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.740 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:04 smithi099 ceph-mon[31715]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' 2024-03-23T23:35:04.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:04 smithi149 ceph-mon[36083]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:35:04.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:04 smithi149 ceph-mon[36083]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T23:35:04.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:04 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:04 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:04.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:04 smithi149 ceph-mon[36083]: from='osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451]' entity='osd.0' 2024-03-23T23:35:05.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] boot 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T23:35:05.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:05.609 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T23:35:05.610 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T23:35:05.610 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-23T23:35:05.610 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-23T23:35:05.610 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.017 s, 617 MB/s 2024-03-23T23:35:05.610 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-23T23:35:05.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:35:05.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:35:05.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:05.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:05.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] boot 2024-03-23T23:35:05.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T23:35:05.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:05.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: osd.0 [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] boot 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T23:35:05.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T23:35:06.260 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-03-23T23:35:06.498 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:35:07.553 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:07 smithi149 ceph-mon[36083]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T23:35:07.553 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:07 smithi149 ceph-mon[36083]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:07 smithi099 ceph-mon[31715]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T23:35:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:07 smithi099 ceph-mon[31715]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:07.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:07 smithi174 ceph-mon[37002]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T23:35:07.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:07 smithi174 ceph-mon[37002]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2298745457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381022b0 msgr2=0x7f1f38102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2298745457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381022b0 0x7f1f38102710 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0099c0 tx=0x7f1f2c009cd0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2298745457 shutdown_connections 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2298745457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f38109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2298745457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f381010b0 0x7f1f381014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2298745457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381022b0 0x7f1f38102710 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2298745457 >> 172.21.15.149:0/2298745457 conn(0x7f1f380fc690 msgr2=0x7f1f380feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.043+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2298745457 shutdown_connections 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2298745457 wait complete. 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 Processor -- start 2024-03-23T23:35:08.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 -- start start 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f381022b0 0x7f1f38197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f3819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3810cd90 con 0x7f1f381022b0 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3810cc10 con 0x7f1f381010b0 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.044+0000 7f1f3d41d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3810cf10 con 0x7f1f38102c50 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:49656/0 (socket says 172.21.15.149:49656) 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 -- 172.21.15.149:0/2988473997 learned_addr learned my addr 172.21.15.149:0/2988473997 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f377fe700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f3819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 -- 172.21.15.149:0/2988473997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 msgr2=0x7f1f3819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f3819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.047 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 -- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f381022b0 msgr2=0x7f1f38197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f381022b0 0x7f1f38197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 -- 172.21.15.149:0/2988473997 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3806a8d0 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f377fe700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f3819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f36ffd700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b500 tx=0x7f1f2800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28015020 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3806abc0 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f28011d10 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3806b1d0 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.045+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800f2e0 con 0x7f1f381010b0 2024-03-23T23:35:08.048 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.046+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38068e50 con 0x7f1f381010b0 2024-03-23T23:35:08.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.047+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f1f28018020 con 0x7f1f381010b0 2024-03-23T23:35:08.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.048+0000 7f1f1ffff700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 0x7f1f200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:08.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.048+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f1f280ccb70 con 0x7f1f381010b0 2024-03-23T23:35:08.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.048+0000 7f1f367fc700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 0x7f1f200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:08.049 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.049+0000 7f1f367fc700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 0x7f1f200a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003790 tx=0x7f1f2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:08.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.052+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f2801f020 con 0x7f1f381010b0 2024-03-23T23:35:08.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:08.258+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f3802db10 con 0x7f1f200a2880 2024-03-23T23:35:08.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:08 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:08.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:08 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:08.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:08 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:08.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:08 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:08.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:08 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:08.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:08 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:08.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:08 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:08.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:08 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:08.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:08 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:09.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:09.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi099 2024-03-23T23:35:09.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.836 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi099 to 17091M 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:09 smithi149 ceph-mon[36083]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:09.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi099 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi099 to 17091M 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:09 smithi099 ceph-mon[31715]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi099 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:09.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi099 to 17091M 2024-03-23T23:35:09.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:09.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:09.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:09.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:09 smithi174 ceph-mon[37002]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/2077558878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]': finished 2024-03-23T23:35:12.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T23:35:12.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:12.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:11 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/379287848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/2077558878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]': finished 2024-03-23T23:35:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T23:35:12.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:12.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:11 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/379287848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:12.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:12.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/2077558878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]: dispatch 2024-03-23T23:35:12.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af37efa-acb5-4b3c-bfbd-a43df66c3893"}]': finished 2024-03-23T23:35:12.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T23:35:12.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:12.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:11 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/379287848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:13 smithi099 ceph-mon[31715]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:13 smithi149 ceph-mon[36083]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:13 smithi174 ceph-mon[37002]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:15.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:15 smithi149 ceph-mon[36083]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:15 smithi099 ceph-mon[31715]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:15 smithi174 ceph-mon[37002]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:17 smithi099 ceph-mon[31715]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T23:35:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:17 smithi149 ceph-mon[36083]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T23:35:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:18.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:17 smithi174 ceph-mon[37002]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T23:35:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:18.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:18 smithi149 ceph-mon[36083]: Deploying daemon osd.1 on smithi149 2024-03-23T23:35:19.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:18 smithi174 ceph-mon[37002]: Deploying daemon osd.1 on smithi149 2024-03-23T23:35:19.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:18 smithi099 ceph-mon[31715]: Deploying daemon osd.1 on smithi149 2024-03-23T23:35:19.889 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:19 smithi149 ceph-mon[36083]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:20.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:19 smithi174 ceph-mon[37002]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:20.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:19 smithi099 ceph-mon[31715]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:22.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:21 smithi099 ceph-mon[31715]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:22.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:21 smithi149 ceph-mon[36083]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:21 smithi174 ceph-mon[37002]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:24.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:23 smithi149 ceph-mon[36083]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:24.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:23 smithi099 ceph-mon[31715]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:24.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:23 smithi174 ceph-mon[37002]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:24.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:24.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:25.995 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:25 smithi149 ceph-mon[36083]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:25 smithi099 ceph-mon[31715]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:25 smithi174 ceph-mon[37002]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:26.648 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.648+0000 7f1f1ffff700 1 -- 172.21.15.149:0/2988473997 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f3802db10 con 0x7f1f200a2880 2024-03-23T23:35:26.649 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2024-03-23T23:35:26.651 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 msgr2=0x7f1f200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:26.651 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 0x7f1f200a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003790 tx=0x7f1f2c000bc0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.651 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 msgr2=0x7f1f381973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:26.651 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800b500 tx=0x7f1f2800b8c0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.651 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 shutdown_connections 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1f200a2880 0x7f1f200a4d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38102c50 0x7f1f3819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f381010b0 0x7f1f381973d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 --2- 172.21.15.149:0/2988473997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f381022b0 0x7f1f38197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 >> 172.21.15.149:0/2988473997 conn(0x7f1f380fc690 msgr2=0x7f1f381055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 shutdown_connections 2024-03-23T23:35:26.652 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:35:26.650+0000 7f1f3d41d700 1 -- 172.21.15.149:0/2988473997 wait complete. 2024-03-23T23:35:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:26.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:27.293 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.1.service 2024-03-23T23:35:27.295 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-03-23T23:35:27.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T23:35:27.537 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:35:27.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:27 smithi174 ceph-mon[37002]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:27.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:27 smithi174 ceph-mon[37002]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:27.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:27 smithi174 ceph-mon[37002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:28.098 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:27 smithi149 ceph-mon[36083]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:28.098 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:27 smithi149 ceph-mon[36083]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:28.098 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:27 smithi149 ceph-mon[36083]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:28.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:27 smithi099 ceph-mon[31715]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:28.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:27 smithi099 ceph-mon[31715]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:28.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:27 smithi099 ceph-mon[31715]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T23:35:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:28 smithi099 ceph-mon[31715]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T23:35:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:28 smithi099 ceph-mon[31715]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T23:35:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:28 smithi099 ceph-mon[31715]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:28 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:28 smithi099 ceph-mon[31715]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:28 smithi149 ceph-mon[36083]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T23:35:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:28 smithi149 ceph-mon[36083]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T23:35:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:28 smithi149 ceph-mon[36083]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:28 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:28 smithi149 ceph-mon[36083]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:28 smithi174 ceph-mon[37002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T23:35:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:28 smithi174 ceph-mon[37002]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T23:35:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:28 smithi174 ceph-mon[37002]: from='osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:28 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:28 smithi174 ceph-mon[37002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131912 s, 795 MB/s 2024-03-23T23:35:29.342 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-23T23:35:29.543 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1[40667]: 2024-03-23T23:35:29.277+0000 7f3b852ff700 -1 osd.1 0 waiting for initial osdmap 2024-03-23T23:35:29.543 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1[40667]: 2024-03-23T23:35:29.280+0000 7f3b7d0ea700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:35:29.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:29 smithi174 ceph-mon[37002]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:29.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:29 smithi174 ceph-mon[37002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:35:29.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:29 smithi174 ceph-mon[37002]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T23:35:29.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:29 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:29.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:29 smithi174 ceph-mon[37002]: from='osd.1 ' entity='osd.1' 2024-03-23T23:35:29.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-03-23T23:35:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:29 smithi099 ceph-mon[31715]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:29 smithi099 ceph-mon[31715]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:35:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:29 smithi099 ceph-mon[31715]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T23:35:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:29 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:29 smithi099 ceph-mon[31715]: from='osd.1 ' entity='osd.1' 2024-03-23T23:35:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-mon[36083]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-mon[36083]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:35:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-mon[36083]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T23:35:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:29 smithi149 ceph-mon[36083]: from='osd.1 ' entity='osd.1' 2024-03-23T23:35:30.207 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] boot 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:30.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:30.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:30.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:30.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:30.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:35:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] boot 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: osd.1 [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] boot 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:31.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:31.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:31.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 -- 172.21.15.174:0/580788819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4101a80 msgr2=0x7f50f4101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/580788819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4101a80 0x7f50f4101ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f50e400aaa0 tx=0x7f50e400adb0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 -- 172.21.15.174:0/580788819 shutdown_connections 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/580788819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50f4102420 0x7f50f4108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/580788819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/580788819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4101a80 0x7f50f4101ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.720+0000 7f50f8cc1700 1 -- 172.21.15.174:0/580788819 >> 172.21.15.174:0/580788819 conn(0x7f50f40fbe20 msgr2=0x7f50f40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- 172.21.15.174:0/580788819 shutdown_connections 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- 172.21.15.174:0/580788819 wait complete. 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 Processor -- start 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- start start 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f41961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50f4101a80 0x7f50f4196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 0x7f50f419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f410bc30 con 0x7f50f4101a80 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f410bab0 con 0x7f50f4100880 2024-03-23T23:35:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.721+0000 7f50f8cc1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f410bdb0 con 0x7f50f4102420 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f41961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 0x7f50f419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f41961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.174:57314/0 (socket says 172.21.15.174:57314) 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f259c700 1 -- 172.21.15.174:0/2473288301 learned_addr learned my addr 172.21.15.174:0/2473288301 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 -- 172.21.15.174:0/2473288301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 msgr2=0x7f50f41961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f41961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 -- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50f4101a80 msgr2=0x7f50f4196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50f4101a80 0x7f50f4196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 -- 172.21.15.174:0/2473288301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f419de40 con 0x7f50f4102420 2024-03-23T23:35:31.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f2d9d700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 0x7f50f419d7a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00be50 tx=0x7f50ec009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec019070 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f419e130 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50ec00cc10 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec012670 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.722+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f419e6a0 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.723+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4068e50 con 0x7f50f4102420 2024-03-23T23:35:31.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.724+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f50ec01c070 con 0x7f50f4102420 2024-03-23T23:35:31.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.725+0000 7f50e37fe700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 0x7f50dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:31.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.725+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f50ec0cc7b0 con 0x7f50f4102420 2024-03-23T23:35:31.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.725+0000 7f50f259c700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 0x7f50dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:31.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.726+0000 7f50f259c700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 0x7f50dc0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50e800bd70 tx=0x7f50e800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:31.729 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.730+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f50ec00cdc0 con 0x7f50f4102420 2024-03-23T23:35:31.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:31 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi149 2024-03-23T23:35:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:31 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-23T23:35:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:31 smithi174 ceph-mon[37002]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:31.938 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:31.938+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f50f40625d0 con 0x7f50dc0a2800 2024-03-23T23:35:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:31 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi149 2024-03-23T23:35:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:31 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-23T23:35:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:31 smithi099 ceph-mon[31715]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:32.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:31 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi149 2024-03-23T23:35:32.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:31 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi149 to 5807M 2024-03-23T23:35:32.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:31 smithi149 ceph-mon[36083]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-23T23:35:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:32 smithi099 ceph-mon[31715]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T23:35:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:32 smithi099 ceph-mon[31715]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:32 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:32 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:32 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:33.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:32 smithi149 ceph-mon[36083]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T23:35:33.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:32 smithi149 ceph-mon[36083]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:33.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:32 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:33.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:32 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:33.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:32 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:33.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:32 smithi174 ceph-mon[37002]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T23:35:33.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:32 smithi174 ceph-mon[37002]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:33.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:32 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:33.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:32 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:33.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:32 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:33.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:33 smithi174 ceph-mon[37002]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:33 smithi099 ceph-mon[31715]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:34.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:33 smithi149 ceph-mon[36083]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:35.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:34 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/3044778298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:34 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:34 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]': finished 2024-03-23T23:35:35.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:34 smithi099 ceph-mon[31715]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T23:35:35.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:35.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:34 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/3044778298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:34 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:34 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]': finished 2024-03-23T23:35:35.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:34 smithi149 ceph-mon[36083]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T23:35:35.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:35.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:34 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/3044778298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:34 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]: dispatch 2024-03-23T23:35:35.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:34 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6afadce7-6cd9-439a-8916-36eb8eb44c99"}]': finished 2024-03-23T23:35:35.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:34 smithi174 ceph-mon[37002]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T23:35:35.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:35 smithi099 ceph-mon[31715]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:35 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/1219087759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:36.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:35 smithi149 ceph-mon[36083]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:36.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:35 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/1219087759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:35 smithi174 ceph-mon[37002]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:35 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/1219087759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:35:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:37 smithi099 ceph-mon[31715]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:38.136 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:37 smithi174 ceph-mon[37002]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:37 smithi149 ceph-mon[36083]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:39 smithi099 ceph-mon[31715]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:40.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:39 smithi149 ceph-mon[36083]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:40.153 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:39 smithi174 ceph-mon[37002]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:41 smithi149 ceph-mon[36083]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T23:35:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:42.169 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:41 smithi174 ceph-mon[37002]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:42.169 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T23:35:42.169 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:41 smithi099 ceph-mon[31715]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T23:35:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:43.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:42 smithi149 ceph-mon[36083]: Deploying daemon osd.2 on smithi174 2024-03-23T23:35:43.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:42 smithi174 ceph-mon[37002]: Deploying daemon osd.2 on smithi174 2024-03-23T23:35:43.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:42 smithi099 ceph-mon[31715]: Deploying daemon osd.2 on smithi174 2024-03-23T23:35:44.129 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:43 smithi174 ceph-mon[37002]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:44.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:43 smithi149 ceph-mon[36083]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:44.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:43 smithi099 ceph-mon[31715]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:46.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:45 smithi149 ceph-mon[36083]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:45 smithi174 ceph-mon[37002]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:45 smithi099 ceph-mon[31715]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:47 smithi149 ceph-mon[36083]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:47 smithi174 ceph-mon[37002]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:47 smithi099 ceph-mon[31715]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:49.824 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:49 smithi174 ceph-mon[37002]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:49.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:49 smithi149 ceph-mon[36083]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:49.920 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-03-23T23:35:49.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.920+0000 7f50e37fe700 1 -- 172.21.15.174:0/2473288301 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f50f40625d0 con 0x7f50dc0a2800 2024-03-23T23:35:49.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 msgr2=0x7f50dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:49.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 0x7f50dc0a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50e800bd70 tx=0x7f50e800b4a0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 msgr2=0x7f50f419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 0x7f50f419d7a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00be50 tx=0x7f50ec009ed0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 shutdown_connections 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f50dc0a2800 0x7f50dc0a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4102420 0x7f50f419d7a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f4100880 0x7f50f41961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 --2- 172.21.15.174:0/2473288301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50f4101a80 0x7f50f4196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.922+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 >> 172.21.15.174:0/2473288301 conn(0x7f50f40fbe20 msgr2=0x7f50f4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.923+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 shutdown_connections 2024-03-23T23:35:49.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:35:49.923+0000 7f50f8cc1700 1 -- 172.21.15.174:0/2473288301 wait complete. 2024-03-23T23:35:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:49 smithi099 ceph-mon[31715]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:50.506 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.2.service 2024-03-23T23:35:50.508 INFO:tasks.cephadm:Deploying osd.3 on smithi099 with /dev/vg_nvme/lv_3... 2024-03-23T23:35:50.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T23:35:50.578 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:35:50.550+0000 7fdb96a5f7c0 -1 osd.2 0 log_to_monitors true 2024-03-23T23:35:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:50 smithi149 ceph-mon[36083]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:51.155 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:50 smithi174 ceph-mon[37002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:51.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:50 smithi099 ceph-mon[31715]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:52.083 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:51 smithi174 ceph-mon[37002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:51 smithi149 ceph-mon[36083]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:52.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:52.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T23:35:52.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T23:35:52.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:52.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: from='osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:52.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:51 smithi099 ceph-mon[31715]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:35:53.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:35:52.704+0000 7fdb8c8c5700 -1 osd.2 0 waiting for initial osdmap 2024-03-23T23:35:53.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:35:52.710+0000 7fdb83eaf700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:53.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:53 smithi174 ceph-mon[37002]: from='osd.2 ' entity='osd.2' 2024-03-23T23:35:53.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:35:53.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T23:35:53.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:53.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:53 smithi099 ceph-mon[31715]: from='osd.2 ' entity='osd.2' 2024-03-23T23:35:53.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:35:53.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T23:35:53.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:53.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-23T23:35:53.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:53 smithi149 ceph-mon[36083]: from='osd.2 ' entity='osd.2' 2024-03-23T23:35:53.851 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T23:35:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T23:35:53.852 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-23T23:35:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-23T23:35:53.852 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0341318 s, 307 MB/s 2024-03-23T23:35:53.852 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi174 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi174 to 17080M 2024-03-23T23:35:54.317 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] boot 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi174 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi174 to 17080M 2024-03-23T23:35:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] boot 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi174 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi174 to 17080M 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: osd.2 [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] boot 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:35:54.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:54.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-03-23T23:35:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:35:55.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:55 smithi099 ceph-mon[31715]: pgmap v92: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:55.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:55.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:55.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:55 smithi099 ceph-mon[31715]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T23:35:55.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:55.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:55 smithi149 ceph-mon[36083]: pgmap v92: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:55.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:55.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:55.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:55 smithi149 ceph-mon[36083]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T23:35:55.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:55.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:55 smithi174 ceph-mon[37002]: pgmap v92: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:55.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:55.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:55.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:55 smithi174 ceph-mon[37002]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T23:35:55.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:35:57.200 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:35:57.200 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:57.200 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:57.200 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48815]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:57.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 -- 172.21.15.99:0/1168745990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1021d0 msgr2=0x7f7edc1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/1168745990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1021d0 0x7f7edc1025f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0099c0 tx=0x7f7ecc009cd0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 -- 172.21.15.99:0/1168745990 shutdown_connections 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/1168745990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/1168745990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1021d0 0x7f7edc1025f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/1168745990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1033d0 0x7f7edc103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 -- 172.21.15.99:0/1168745990 >> 172.21.15.99:0/1168745990 conn(0x7f7edc0fd730 msgr2=0x7f7edc0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 -- 172.21.15.99:0/1168745990 shutdown_connections 2024-03-23T23:35:57.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.362+0000 7f7ee2e00700 1 -- 172.21.15.99:0/1168745990 wait complete. 2024-03-23T23:35:57.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 Processor -- start 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 -- start start 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1021d0 0x7f7edc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 0x7f7edc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc10ddc0 con 0x7f7edc1033d0 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc10dc40 con 0x7f7edc1021d0 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.363+0000 7f7ee2e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc10df40 con 0x7f7edc103d70 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7edb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 0x7f7edc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7edb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 0x7f7edc0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35610/0 (socket says 172.21.15.99:35610) 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7edb400700 1 -- 172.21.15.99:0/2204272862 learned_addr learned my addr 172.21.15.99:0/2204272862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7edbe00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1021d0 0x7f7edc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:57.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 -- 172.21.15.99:0/2204272862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1021d0 msgr2=0x7f7edc075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1021d0 0x7f7edc075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 -- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 msgr2=0x7f7edc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 0x7f7edc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 -- 172.21.15.99:0/2204272862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc075060 con 0x7f7edc103d70 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.364+0000 7f7ee0e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc074b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400efc0 tx=0x7f7ed400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.365+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40092e0 con 0x7f7edc103d70 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.365+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0752f0 con 0x7f7edc103d70 2024-03-23T23:35:57.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.365+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc1a6e50 con 0x7f7edc103d70 2024-03-23T23:35:57.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.365+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ed4004bb0 con 0x7f7edc103d70 2024-03-23T23:35:57.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.365+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4004040 con 0x7f7edc103d70 2024-03-23T23:35:57.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.366+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc068e50 con 0x7f7edc103d70 2024-03-23T23:35:57.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.368+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7ed401b030 con 0x7f7edc103d70 2024-03-23T23:35:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.369+0000 7f7ed8c00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 0x7f7ec00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:35:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.369+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f7ed40cd240 con 0x7f7edc103d70 2024-03-23T23:35:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.369+0000 7f7edbe00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 0x7f7ec00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:35:57.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.370+0000 7f7edbe00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 0x7f7ec00a4d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc003790 tx=0x7f7ecc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:35:57.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.374+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ed4022020 con 0x7f7edc103d70 2024-03-23T23:35:57.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:35:57.610+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7edc0625d0 con 0x7f7ec00a2890 2024-03-23T23:35:57.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:57.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:57 smithi099 ceph-mon[31715]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T23:35:57.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:57 smithi099 ceph-mon[31715]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:57.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:57.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:57 smithi149 ceph-mon[36083]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T23:35:57.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:57 smithi149 ceph-mon[36083]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:57.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:35:57.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:57 smithi174 ceph-mon[37002]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T23:35:57.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:57 smithi174 ceph-mon[37002]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:35:58.111 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48856]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:35:58.111 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:58.111 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48856]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:58.111 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:35:57 smithi099 sudo[48856]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:58.201 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:57 smithi149 sudo[44383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:35:58.201 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:57 smithi149 sudo[44383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:58.201 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:57 smithi149 sudo[44383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:58.201 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:57 smithi149 sudo[44383]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:58.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:58.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:58 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:58.643 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:58 smithi149 sudo[44386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:35:58.644 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:58 smithi149 sudo[44386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:58.644 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:58 smithi149 sudo[44386]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:58.644 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:35:58 smithi149 sudo[44386]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:58.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:35:58 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:58.673 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45803]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:35:58.673 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:58.673 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45803]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:58.673 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45803]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:35:58.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:35:58.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:58.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:35:58 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:35:59.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:35:59.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:59.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45806]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:59.173 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:35:58 smithi174 sudo[45806]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:35:59.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:35:59.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:35:59.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48986]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:35:59.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48986]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:00.044 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:36:00.044 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:36:00.045 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:36:00.045 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:35:59 smithi099 sudo[48998]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:00.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:36:00.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:36:00.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45809]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:36:00.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45809]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:00.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:00 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:00.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:00 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 sudo[45812]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:00.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:00 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:00.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:00.676 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:00.939 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T23:36:00.940 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:36:00.940 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44390]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:36:00.940 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44390]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:01.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T23:36:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T23:36:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44401]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T23:36:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:00 smithi149 sudo[44401]: pam_unix(sudo:session): session closed for user root 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:01 smithi099 ceph-mon[31715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:01 smithi149 ceph-mon[36083]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:01.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:01 smithi174 ceph-mon[37002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1184537110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]': finished 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:02.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:02 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2784811006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1184537110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]': finished 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:02.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:02 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2784811006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1184537110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]: dispatch 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acb0cde2-7e55-4edf-baa5-3f6c46a83299"}]': finished 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:02.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:02 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2784811006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:03.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:03 smithi099 ceph-mon[31715]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:03.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:03 smithi149 ceph-mon[36083]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:03.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:03 smithi174 ceph-mon[37002]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:05.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:05 smithi099 ceph-mon[31715]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:05 smithi149 ceph-mon[36083]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:06.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:05 smithi174 ceph-mon[37002]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:07.795 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:07 smithi099 ceph-mon[31715]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 51 KiB/s wr, 4 op/s 2024-03-23T23:36:08.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:07 smithi149 ceph-mon[36083]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 51 KiB/s wr, 4 op/s 2024-03-23T23:36:08.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:07 smithi174 ceph-mon[37002]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 51 KiB/s wr, 4 op/s 2024-03-23T23:36:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:09 smithi099 ceph-mon[31715]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-03-23T23:36:10.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:09 smithi149 ceph-mon[36083]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-03-23T23:36:10.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:09 smithi174 ceph-mon[37002]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-03-23T23:36:11.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T23:36:11.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:11.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T23:36:11.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:11.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T23:36:11.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:11.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:11 smithi099 ceph-mon[31715]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1010 B/s rd, 29 KiB/s wr, 2 op/s 2024-03-23T23:36:11.951 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:11 smithi099 ceph-mon[31715]: Deploying daemon osd.3 on smithi099 2024-03-23T23:36:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:11 smithi149 ceph-mon[36083]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1010 B/s rd, 29 KiB/s wr, 2 op/s 2024-03-23T23:36:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:11 smithi149 ceph-mon[36083]: Deploying daemon osd.3 on smithi099 2024-03-23T23:36:12.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:11 smithi174 ceph-mon[37002]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1010 B/s rd, 29 KiB/s wr, 2 op/s 2024-03-23T23:36:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:11 smithi174 ceph-mon[37002]: Deploying daemon osd.3 on smithi099 2024-03-23T23:36:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:13 smithi099 ceph-mon[31715]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-23T23:36:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:13 smithi149 ceph-mon[36083]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-23T23:36:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:13 smithi174 ceph-mon[37002]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 828 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-23T23:36:15.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:15 smithi099 ceph-mon[31715]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:15 smithi149 ceph-mon[36083]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:15 smithi174 ceph-mon[37002]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:17 smithi099 ceph-mon[31715]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:17 smithi149 ceph-mon[36083]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:17 smithi174 ceph-mon[37002]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-23T23:36:20.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:19 smithi099 ceph-mon[31715]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:20.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:19 smithi149 ceph-mon[36083]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:19 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:19 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:19 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:20.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:19 smithi174 ceph-mon[37002]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:22.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:21 smithi099 ceph-mon[31715]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:21 smithi149 ceph-mon[36083]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:22.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:21 smithi174 ceph-mon[37002]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:23.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:23.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:23 smithi099 ceph-mon[31715]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:23.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:23 smithi149 ceph-mon[36083]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:23 smithi174 ceph-mon[37002]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T23:36:24.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.207+0000 7f7ed8c00700 1 -- 172.21.15.99:0/2204272862 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7edc0625d0 con 0x7f7ec00a2890 2024-03-23T23:36:24.209 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 3 on host 'smithi099' 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 msgr2=0x7f7ec00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 0x7f7ec00a4d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc003790 tx=0x7f7ecc005fb0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 msgr2=0x7f7edc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc074b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400efc0 tx=0x7f7ed400c3a0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 shutdown_connections 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ec00a2890 0x7f7ec00a4d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc103d70 0x7f7edc074b20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7edc1021d0 0x7f7edc075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 --2- 172.21.15.99:0/2204272862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7edc1033d0 0x7f7edc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:24.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 >> 172.21.15.99:0/2204272862 conn(0x7f7edc0fd730 msgr2=0x7f7edc1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:36:24.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.211+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 shutdown_connections 2024-03-23T23:36:24.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:36:24.212+0000 7f7ee2e00700 1 -- 172.21.15.99:0/2204272862 wait complete. 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:24.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:25.122 DEBUG:teuthology.orchestra.run.smithi099:osd.3> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.3.service 2024-03-23T23:36:25.127 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-03-23T23:36:25.127 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T23:36:25.374 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:36:25.616 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: from='osd.3 ' entity='osd.3' 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] boot 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T23:36:25.617 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:36:25.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T23:36:25.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:25.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: from='osd.3 ' entity='osd.3' 2024-03-23T23:36:25.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] boot 2024-03-23T23:36:25.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T23:36:25.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: from='osd.3 ' entity='osd.3' 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: osd.3 [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] boot 2024-03-23T23:36:25.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T23:36:25.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T23:36:26.819 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:26 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:36:26.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:26 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:36:26.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:26 smithi149 ceph-mon[36083]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T23:36:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:26 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:36:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:26 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:36:26.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:26 smithi099 ceph-mon[31715]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T23:36:26.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:26 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:36:26.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:26 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:36:26.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:26 smithi174 ceph-mon[37002]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T23:36:27.410 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T23:36:27.411 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T23:36:27.411 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-23T23:36:27.411 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-23T23:36:27.411 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113446 s, 924 MB/s 2024-03-23T23:36:27.411 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-23T23:36:27.757 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:27 smithi149 ceph-mon[36083]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:27.757 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:27 smithi149 ceph-mon[36083]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T23:36:27.801 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:27 smithi099 ceph-mon[31715]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:27.802 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:27 smithi099 ceph-mon[31715]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T23:36:27.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:27 smithi174 ceph-mon[37002]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:27.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:27 smithi174 ceph-mon[37002]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T23:36:28.101 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-03-23T23:36:28.348 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:36:29.817 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:29 smithi149 ceph-mon[36083]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T23:36:29.817 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:29 smithi149 ceph-mon[36083]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:29.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:29 smithi099 ceph-mon[31715]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T23:36:29.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:29 smithi099 ceph-mon[31715]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:29.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:29 smithi174 ceph-mon[37002]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T23:36:29.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:29 smithi174 ceph-mon[37002]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:30.005 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 -- 172.21.15.149:0/3557085595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde88102c60 msgr2=0x7fde88109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3557085595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde88102c60 0x7fde88109510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fde8000b0d0 tx=0x7fde8000b3e0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 -- 172.21.15.149:0/3557085595 shutdown_connections 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3557085595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde88102c60 0x7fde88109510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3557085595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde881010c0 0x7fde881014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3557085595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 0x7fde88102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 -- 172.21.15.149:0/3557085595 >> 172.21.15.149:0/3557085595 conn(0x7fde880fc680 msgr2=0x7fde880feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 -- 172.21.15.149:0/3557085595 shutdown_connections 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.002+0000 7fde8cafa700 1 -- 172.21.15.149:0/3557085595 wait complete. 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 Processor -- start 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 -- start start 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:30.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 0x7fde8819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:30.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde88102c60 0x7fde881a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:30.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8810cd30 con 0x7fde88102c60 2024-03-23T23:36:30.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8810cbb0 con 0x7fde881010c0 2024-03-23T23:36:30.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8cafa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8810ceb0 con 0x7fde881022c0 2024-03-23T23:36:30.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:48246/0 (socket says 172.21.15.149:48246) 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 -- 172.21.15.149:0/3200113107 learned_addr learned my addr 172.21.15.149:0/3200113107 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde85d9b700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 0x7fde8819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 -- 172.21.15.149:0/3200113107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 msgr2=0x7fde8819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 0x7fde8819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 -- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde88102c60 msgr2=0x7fde881a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:30.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde88102c60 0x7fde881a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:30.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 -- 172.21.15.149:0/3200113107 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde881a2240 con 0x7fde881010c0 2024-03-23T23:36:30.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.003+0000 7fde8659c700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde7800b500 tx=0x7fde7800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:36:30.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.004+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78015020 con 0x7fde881010c0 2024-03-23T23:36:30.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.004+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde881a2530 con 0x7fde881010c0 2024-03-23T23:36:30.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.004+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde78011d10 con 0x7fde881010c0 2024-03-23T23:36:30.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.004+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800f2e0 con 0x7fde881010c0 2024-03-23T23:36:30.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.004+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde881a2af0 con 0x7fde881010c0 2024-03-23T23:36:30.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.005+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde88068e50 con 0x7fde881010c0 2024-03-23T23:36:30.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.006+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fde78018020 con 0x7fde881010c0 2024-03-23T23:36:30.014 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.006+0000 7fde777fe700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 0x7fde700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:30.014 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.006+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fde780cd4b0 con 0x7fde881010c0 2024-03-23T23:36:30.014 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.007+0000 7fde85d9b700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 0x7fde700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:30.014 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.008+0000 7fde85d9b700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 0x7fde700a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005fd0 tx=0x7fde7c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:36:30.014 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.010+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde7801f020 con 0x7fde881010c0 2024-03-23T23:36:30.225 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:30.225+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fde8806afc0 con 0x7fde700a2890 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:30.885 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:30.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:31.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:31.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:31 smithi149 ceph-mon[36083]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:31.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:31 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi099 2024-03-23T23:36:31.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:31 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi099 to 8545M 2024-03-23T23:36:31.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:31 smithi149 ceph-mon[36083]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:31 smithi174 ceph-mon[37002]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:31 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi099 2024-03-23T23:36:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:31 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi099 to 8545M 2024-03-23T23:36:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:31 smithi174 ceph-mon[37002]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:31 smithi099 ceph-mon[31715]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:31 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi099 2024-03-23T23:36:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:31 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi099 to 8545M 2024-03-23T23:36:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:31 smithi099 ceph-mon[31715]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/1192188003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]': finished 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:33 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/81851207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/1192188003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]': finished 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:33 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/81851207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:34.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/1192188003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]: dispatch 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d32abb2-0d09-4f68-a5a0-2a823ec6a012"}]': finished 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:33 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/81851207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:35 smithi099 ceph-mon[31715]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T23:36:36.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:35 smithi149 ceph-mon[36083]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T23:36:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:35 smithi174 ceph-mon[37002]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-23T23:36:38.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:37 smithi149 ceph-mon[36083]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:36:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:37 smithi099 ceph-mon[31715]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:36:38.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:37 smithi174 ceph-mon[37002]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:36:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:39 smithi099 ceph-mon[31715]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-23T23:36:40.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:39 smithi149 ceph-mon[36083]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-23T23:36:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:39 smithi174 ceph-mon[37002]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-23T23:36:41.073 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:40 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T23:36:41.073 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:40 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:40 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T23:36:41.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:40 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:40 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T23:36:41.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:40 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:42.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:41 smithi149 ceph-mon[36083]: Deploying daemon osd.4 on smithi149 2024-03-23T23:36:42.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:41 smithi149 ceph-mon[36083]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:36:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:41 smithi099 ceph-mon[31715]: Deploying daemon osd.4 on smithi149 2024-03-23T23:36:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:41 smithi099 ceph-mon[31715]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:36:42.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:41 smithi174 ceph-mon[37002]: Deploying daemon osd.4 on smithi149 2024-03-23T23:36:42.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:41 smithi174 ceph-mon[37002]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:36:44.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:43 smithi149 ceph-mon[36083]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:43 smithi099 ceph-mon[31715]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:44.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:43 smithi174 ceph-mon[37002]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:46.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:45 smithi099 ceph-mon[31715]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:45 smithi149 ceph-mon[36083]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:46.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:45 smithi174 ceph-mon[37002]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:47.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:47.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:47.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:48.078 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:47 smithi149 ceph-mon[36083]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:48.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:47 smithi099 ceph-mon[31715]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:47 smithi174 ceph-mon[37002]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:49.671 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.672+0000 7fde777fe700 1 -- 172.21.15.149:0/3200113107 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fde8806afc0 con 0x7fde700a2890 2024-03-23T23:36:49.672 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-03-23T23:36:49.674 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 msgr2=0x7fde700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:49.674 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 0x7fde700a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005fd0 tx=0x7fde7c00a560 comp rx=0 tx=0).stop 2024-03-23T23:36:49.674 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 msgr2=0x7fde8819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde7800b500 tx=0x7fde7800b8c0 comp rx=0 tx=0).stop 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 shutdown_connections 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fde700a2890 0x7fde700a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde88102c60 0x7fde881a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde881010c0 0x7fde8819a5e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 --2- 172.21.15.149:0/3200113107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde881022c0 0x7fde8819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.674+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 >> 172.21.15.149:0/3200113107 conn(0x7fde880fc680 msgr2=0x7fde8806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.675+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 shutdown_connections 2024-03-23T23:36:49.675 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:36:49.675+0000 7fde8cafa700 1 -- 172.21.15.149:0/3200113107 wait complete. 2024-03-23T23:36:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:49 smithi149 ceph-mon[36083]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:49 smithi149 ceph-mon[36083]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:49 smithi149 ceph-mon[36083]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.225 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:49 smithi174 ceph-mon[37002]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:50.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:49 smithi174 ceph-mon[37002]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.226 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:49 smithi174 ceph-mon[37002]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.341 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.4.service 2024-03-23T23:36:50.343 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-03-23T23:36:50.343 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T23:36:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:49 smithi099 ceph-mon[31715]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:49 smithi099 ceph-mon[31715]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:49 smithi099 ceph-mon[31715]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T23:36:50.597 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:51.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:51 smithi149 ceph-mon[36083]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T23:36:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.740 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T23:36:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:51 smithi174 ceph-mon[37002]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:51.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:51.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T23:36:51.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T23:36:51.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:51.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:36:51.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:51 smithi099 ceph-mon[31715]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:52.345 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:36:52.060+0000 7f7720e67700 -1 osd.4 0 waiting for initial osdmap 2024-03-23T23:36:52.345 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:36:52.065+0000 7f7718451700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114955 s, 912 MB/s 2024-03-23T23:36:52.711 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-23T23:36:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-mon[36083]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:36:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-mon[36083]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T23:36:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:52 smithi149 ceph-mon[36083]: from='osd.4 ' entity='osd.4' 2024-03-23T23:36:52.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:52 smithi174 ceph-mon[37002]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:36:52.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:52 smithi174 ceph-mon[37002]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T23:36:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:52 smithi174 ceph-mon[37002]: from='osd.4 ' entity='osd.4' 2024-03-23T23:36:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:52 smithi099 ceph-mon[31715]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:36:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:52 smithi099 ceph-mon[31715]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T23:36:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:52 smithi099 ceph-mon[31715]: from='osd.4 ' entity='osd.4' 2024-03-23T23:36:53.360 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-03-23T23:36:53.594 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] boot 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T23:36:53.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:53.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] boot 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T23:36:53.897 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: osd.4 [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] boot 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T23:36:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- 172.21.15.174:0/1882386560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c103e50 msgr2=0x7f201c10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 --2- 172.21.15.174:0/1882386560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c103e50 0x7f201c10a700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f201400b0a0 tx=0x7f201400b3b0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- 172.21.15.174:0/1882386560 shutdown_connections 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 --2- 172.21.15.174:0/1882386560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c103e50 0x7f201c10a700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 --2- 172.21.15.174:0/1882386560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f201c1004d0 0x7f201c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 --2- 172.21.15.174:0/1882386560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1034b0 0x7f201c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- 172.21.15.174:0/1882386560 >> 172.21.15.174:0/1882386560 conn(0x7f201c0fbe30 msgr2=0x7f201c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:36:55.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- 172.21.15.174:0/1882386560 shutdown_connections 2024-03-23T23:36:55.048 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- 172.21.15.174:0/1882386560 wait complete. 2024-03-23T23:36:55.048 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 Processor -- start 2024-03-23T23:36:55.048 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.047+0000 7f2021900700 1 -- start start 2024-03-23T23:36:55.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1004d0 0x7f201c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:55.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:55.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f201c103e50 0x7f201c19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:55.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c10de90 con 0x7f201c103e50 2024-03-23T23:36:55.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c10dd10 con 0x7f201c1004d0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c10e010 con 0x7f201c1034b0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:59998/0 (socket says 172.21.15.174:59998) 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 -- 172.21.15.174:0/3978732514 learned_addr learned my addr 172.21.15.174:0/3978732514 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201affd700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1004d0 0x7f201c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 -- 172.21.15.174:0/3978732514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1004d0 msgr2=0x7f201c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1004d0 0x7f201c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 -- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f201c103e50 msgr2=0x7f201c19f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f201c103e50 0x7f201c19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 -- 172.21.15.174:0/3978732514 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c1a0080 con 0x7f201c1034b0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f201a7fc700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f201000d9b0 tx=0x7f201000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010017070 con 0x7f201c1034b0 2024-03-23T23:36:55.050 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201c1a0370 con 0x7f201c1034b0 2024-03-23T23:36:55.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2010005020 con 0x7f201c1034b0 2024-03-23T23:36:55.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010009bd0 con 0x7f201c1034b0 2024-03-23T23:36:55.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.048+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201c1a08b0 con 0x7f201c1034b0 2024-03-23T23:36:55.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.049+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c068e50 con 0x7f201c1034b0 2024-03-23T23:36:55.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.051+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f20100051d0 con 0x7f201c1034b0 2024-03-23T23:36:55.052 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.051+0000 7f20208fe700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 0x7f20040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:36:55.052 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.051+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f20100cdd50 con 0x7f201c1034b0 2024-03-23T23:36:55.052 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.052+0000 7f201affd700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 0x7f20040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:36:55.052 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.052+0000 7f201affd700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 0x7f20040a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f201c06a6b0 tx=0x7f200c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:36:55.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.055+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2010020020 con 0x7f201c1034b0 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi149 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:55.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:54 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.256 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:36:55.256+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f201c0625d0 con 0x7f20040a2890 2024-03-23T23:36:55.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi149 2024-03-23T23:36:55.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:54 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi149 2024-03-23T23:36:55.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:36:55.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:54 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:55 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:56.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-23T23:36:56.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:56.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T23:36:56.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:56.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:56.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:55 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:56.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi149 to 2903M 2024-03-23T23:36:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T23:36:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:36:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:36:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:55 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:36:57.168 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:56 smithi174 ceph-mon[37002]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:57.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:56 smithi099 ceph-mon[31715]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:57.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:56 smithi149 ceph-mon[36083]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:36:58.349 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:58.350 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/645632830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.350 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.350 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]': finished 2024-03-23T23:36:58.350 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T23:36:58.350 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/645632830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]': finished 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T23:36:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/645632830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]: dispatch 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f4ee3c4-a445-4abf-bf98-25a56828b679"}]': finished 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T23:36:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:36:59.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:58 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/2894846006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:59.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:36:58 smithi099 ceph-mon[31715]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:59.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:58 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/2894846006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:59.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:36:58 smithi149 ceph-mon[36083]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:36:59.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:58 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/2894846006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:36:59.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:36:58 smithi174 ceph-mon[37002]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:01 smithi174 ceph-mon[37002]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:02.097 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:01 smithi149 ceph-mon[36083]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:01 smithi099 ceph-mon[31715]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:03.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:03 smithi174 ceph-mon[37002]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:03 smithi099 ceph-mon[31715]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:04.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:03 smithi149 ceph-mon[36083]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:05.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:05 smithi174 ceph-mon[37002]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:05.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T23:37:05.972 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:05 smithi099 ceph-mon[31715]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T23:37:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:05 smithi149 ceph-mon[36083]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T23:37:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:07.048 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:06 smithi174 ceph-mon[37002]: Deploying daemon osd.5 on smithi174 2024-03-23T23:37:07.078 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:06 smithi099 ceph-mon[31715]: Deploying daemon osd.5 on smithi174 2024-03-23T23:37:07.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:06 smithi149 ceph-mon[36083]: Deploying daemon osd.5 on smithi174 2024-03-23T23:37:08.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:07 smithi099 ceph-mon[31715]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:08.113 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:07 smithi174 ceph-mon[37002]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:07 smithi149 ceph-mon[36083]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:09.990 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:09 smithi174 ceph-mon[37002]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:09 smithi099 ceph-mon[31715]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:10.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:09 smithi149 ceph-mon[36083]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:11.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:11.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:11 smithi099 ceph-mon[31715]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:11.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:11.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:11 smithi149 ceph-mon[36083]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:11.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:11.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:11.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:11 smithi174 ceph-mon[37002]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:13.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:13 smithi174 ceph-mon[37002]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:14.045 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.044+0000 7f20208fe700 1 -- 172.21.15.174:0/3978732514 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f201c0625d0 con 0x7f20040a2890 2024-03-23T23:37:14.045 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 msgr2=0x7f20040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 0x7f20040a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f201c06a6b0 tx=0x7f200c009190 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 msgr2=0x7f201c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f201000d9b0 tx=0x7f201000dcc0 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 shutdown_connections 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20040a2890 0x7f20040a4d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f201c103e50 0x7f201c19f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f201c1004d0 0x7f201c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 --2- 172.21.15.174:0/3978732514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201c1034b0 0x7f201c198960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 >> 172.21.15.174:0/3978732514 conn(0x7f201c0fbe30 msgr2=0x7f201c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 shutdown_connections 2024-03-23T23:37:14.047 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:37:14.047+0000 7f2021900700 1 -- 172.21.15.174:0/3978732514 wait complete. 2024-03-23T23:37:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:13 smithi099 ceph-mon[31715]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:14.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:13 smithi149 ceph-mon[36083]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:14.682 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.5.service 2024-03-23T23:37:14.684 INFO:tasks.cephadm:Deploying osd.6 on smithi099 with /dev/vg_nvme/lv_2... 2024-03-23T23:37:14.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:14 smithi099 ceph-mon[31715]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:15.122 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:14 smithi149 ceph-mon[36083]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:15.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T23:37:15.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:14 smithi174 ceph-mon[37002]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:16.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:16 smithi099 ceph-mon[31715]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T23:37:16.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:16 smithi099 ceph-mon[31715]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T23:37:16.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:16 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:16.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:16 smithi099 ceph-mon[31715]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:16 smithi099 ceph-mon[31715]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:16 smithi149 ceph-mon[36083]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T23:37:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:16 smithi149 ceph-mon[36083]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T23:37:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:16 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:16 smithi149 ceph-mon[36083]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:16 smithi149 ceph-mon[36083]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-mon[37002]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T23:37:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-mon[37002]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T23:37:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-mon[37002]: from='osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-mon[37002]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:37:16.923 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:37:16.580+0000 7f961c0d9700 -1 osd.5 0 waiting for initial osdmap 2024-03-23T23:37:16.923 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:37:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:37:16.587+0000 7f9613ec4700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: from='osd.5 ' entity='osd.5' 2024-03-23T23:37:17.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:17 smithi099 ceph-mon[31715]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:17.267 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:37:17.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T23:37:17.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: from='osd.5 ' entity='osd.5' 2024-03-23T23:37:17.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:17 smithi174 ceph-mon[37002]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: from='osd.5 ' entity='osd.5' 2024-03-23T23:37:17.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:17 smithi149 ceph-mon[36083]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0369734 s, 284 MB/s 2024-03-23T23:37:18.166 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-23T23:37:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:37:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:37:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] boot 2024-03-23T23:37:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi174 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi174 to 8540M 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] boot 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi174 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi174 to 8540M 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: osd.5 [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] boot 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi174 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi174 to 8540M 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:18.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:19.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-03-23T23:37:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:19 smithi099 ceph-mon[31715]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T23:37:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:19 smithi099 ceph-mon[31715]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:19.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:19 smithi149 ceph-mon[36083]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T23:37:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:19 smithi149 ceph-mon[36083]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:19 smithi174 ceph-mon[37002]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T23:37:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:19 smithi174 ceph-mon[37002]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:19.619 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:37:20.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:20 smithi174 ceph-mon[37002]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T23:37:20.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:20 smithi099 ceph-mon[31715]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T23:37:20.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:20 smithi149 ceph-mon[36083]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T23:37:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:21 smithi174 ceph-mon[37002]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T23:37:21.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:21 smithi174 ceph-mon[37002]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:21.468 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:21 smithi099 ceph-mon[31715]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T23:37:21.468 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:21 smithi099 ceph-mon[31715]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:21.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:21 smithi149 ceph-mon[36083]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T23:37:21.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:21 smithi149 ceph-mon[36083]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 -- 172.21.15.99:0/621564968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c102c40 msgr2=0x7f527c105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 --2- 172.21.15.99:0/621564968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c102c40 0x7f527c105030 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5270009990 tx=0x7f5270009ca0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 -- 172.21.15.99:0/621564968 shutdown_connections 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 --2- 172.21.15.99:0/621564968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c10a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 --2- 172.21.15.99:0/621564968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c102c40 0x7f527c105030 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 --2- 172.21.15.99:0/621564968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c105570 0x7f527c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 -- 172.21.15.99:0/621564968 >> 172.21.15.99:0/621564968 conn(0x7f527c0fc650 msgr2=0x7f527c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 -- 172.21.15.99:0/621564968 shutdown_connections 2024-03-23T23:37:21.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.976+0000 7f5284000700 1 -- 172.21.15.99:0/621564968 wait complete. 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.977+0000 7f5284000700 1 Processor -- start 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.977+0000 7f5284000700 1 -- start start 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c102c40 0x7f527c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c105570 0x7f527c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c10dd90 con 0x7f527c107ea0 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c10dc10 con 0x7f527c102c40 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5284000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c10df10 con 0x7f527c105570 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5282000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5282000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42640/0 (socket says 172.21.15.99:42640) 2024-03-23T23:37:21.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5282000700 1 -- 172.21.15.99:0/2597826389 learned_addr learned my addr 172.21.15.99:0/2597826389 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5280c00700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c105570 0x7f527c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5281600700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c102c40 0x7f527c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.978+0000 7f5282000700 1 -- 172.21.15.99:0/2597826389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c105570 msgr2=0x7f527c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5282000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c105570 0x7f527c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5282000700 1 -- 172.21.15.99:0/2597826389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c102c40 msgr2=0x7f527c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5282000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c102c40 0x7f527c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5282000700 1 -- 172.21.15.99:0/2597826389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c1a21f0 con 0x7f527c107ea0 2024-03-23T23:37:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5282000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f527400efc0 tx=0x7f527400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740092e0 con 0x7f527c107ea0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c1a2480 con 0x7f527c107ea0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5274004bb0 con 0x7f527c107ea0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274004280 con 0x7f527c107ea0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.979+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c1a29c0 con 0x7f527c107ea0 2024-03-23T23:37:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.980+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c109a20 con 0x7f527c107ea0 2024-03-23T23:37:21.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.984+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f527401b030 con 0x7f527c107ea0 2024-03-23T23:37:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.985+0000 7f527a000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 0x7f52640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.986+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f52740ce110 con 0x7f527c107ea0 2024-03-23T23:37:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.986+0000 7f5281600700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 0x7f52640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.986+0000 7f5281600700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 0x7f52640a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f527000b2e0 tx=0x7f527000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:37:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:21.989+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5274022020 con 0x7f527c107ea0 2024-03-23T23:37:22.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:22.217+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f527c0625d0 con 0x7f52640a2890 2024-03-23T23:37:22.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:22 smithi174 ceph-mon[37002]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T23:37:22.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:22 smithi099 ceph-mon[31715]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T23:37:22.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:22 smithi149 ceph-mon[36083]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T23:37:23.501 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:23 smithi099 ceph-mon[31715]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:23.501 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:23.501 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:23.502 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:23.502 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:23 smithi099 ceph-mon[31715]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-23T23:37:23.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:23 smithi149 ceph-mon[36083]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:23.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:23.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:23.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:23.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:23 smithi149 ceph-mon[36083]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-23T23:37:23.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:23 smithi174 ceph-mon[37002]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:23.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:23.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:23.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:23.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:23 smithi174 ceph-mon[37002]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-23T23:37:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:25 smithi099 ceph-mon[31715]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:37:26.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:25 smithi149 ceph-mon[36083]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:37:26.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:25 smithi174 ceph-mon[37002]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:37:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]: dispatch 2024-03-23T23:37:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]': finished 2024-03-23T23:37:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:26 smithi099 ceph-mon[31715]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T23:37:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]: dispatch 2024-03-23T23:37:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]': finished 2024-03-23T23:37:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:26 smithi149 ceph-mon[36083]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T23:37:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]: dispatch 2024-03-23T23:37:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2101563802' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a94e089-ed73-4d72-b458-74bf3589a1fd"}]': finished 2024-03-23T23:37:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:26 smithi174 ceph-mon[37002]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T23:37:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:27 smithi099 ceph-mon[31715]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-23T23:37:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:27 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1883366622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:37:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:27 smithi149 ceph-mon[36083]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-23T23:37:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:27 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1883366622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:37:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:27 smithi174 ceph-mon[37002]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-23T23:37:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:27 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1883366622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:37:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:29 smithi099 ceph-mon[31715]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:37:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:29 smithi149 ceph-mon[36083]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:37:30.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:29 smithi174 ceph-mon[37002]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:37:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:31 smithi174 ceph-mon[37002]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:31 smithi099 ceph-mon[31715]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:32.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:31 smithi149 ceph-mon[36083]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:33 smithi099 ceph-mon[31715]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:33 smithi149 ceph-mon[36083]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:34.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:33 smithi174 ceph-mon[37002]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:36.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:35 smithi099 ceph-mon[31715]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:36.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:35 smithi149 ceph-mon[36083]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:36.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:35 smithi174 ceph-mon[37002]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:37.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:36 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T23:37:37.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:36 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:37.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:36 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T23:37:37.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:36 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:37.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:36 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T23:37:37.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:36 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:37 smithi099 ceph-mon[31715]: Deploying daemon osd.6 on smithi099 2024-03-23T23:37:37.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:37 smithi099 ceph-mon[31715]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:38.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:37 smithi149 ceph-mon[36083]: Deploying daemon osd.6 on smithi099 2024-03-23T23:37:38.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:37 smithi149 ceph-mon[36083]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:38.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:37 smithi174 ceph-mon[37002]: Deploying daemon osd.6 on smithi099 2024-03-23T23:37:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:37 smithi174 ceph-mon[37002]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:39.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:39 smithi099 ceph-mon[31715]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:40.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:39 smithi149 ceph-mon[36083]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:40.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:39 smithi174 ceph-mon[37002]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:42.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:41 smithi099 ceph-mon[31715]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:41 smithi149 ceph-mon[36083]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:42.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:41 smithi174 ceph-mon[37002]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:43 smithi099 ceph-mon[31715]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:44.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:43 smithi149 ceph-mon[36083]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:44.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:43 smithi174 ceph-mon[37002]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:45.897 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:45 smithi099 ceph-mon[31715]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:45.897 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:45.897 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:45.897 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:45 smithi149 ceph-mon[36083]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:45 smithi174 ceph-mon[37002]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:48.103 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:47 smithi099 ceph-mon[31715]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:47 smithi149 ceph-mon[36083]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:47 smithi174 ceph-mon[37002]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:49.056 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:48 smithi099 ceph-mon[31715]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.056 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:48 smithi099 ceph-mon[31715]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:48 smithi149 ceph-mon[36083]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:48 smithi149 ceph-mon[36083]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:48 smithi174 ceph-mon[37002]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:48 smithi174 ceph-mon[37002]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T23:37:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:49.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:50.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.075 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T23:37:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T23:37:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:50.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:37:50.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.169 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 6 on host 'smithi099' 2024-03-23T23:37:50.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.167+0000 7f527a000700 1 -- 172.21.15.99:0/2597826389 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f527c0625d0 con 0x7f52640a2890 2024-03-23T23:37:50.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 msgr2=0x7f52640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 0x7f52640a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f527000b2e0 tx=0x7f527000b880 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 msgr2=0x7f527c1a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f527400efc0 tx=0x7f527400c3a0 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 shutdown_connections 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f52640a2890 0x7f52640a4d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f527c107ea0 0x7f527c1a1b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c102c40 0x7f527c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 --2- 172.21.15.99:0/2597826389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c105570 0x7f527c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.170+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 >> 172.21.15.99:0/2597826389 conn(0x7f527c0fc650 msgr2=0x7f527c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.171+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 shutdown_connections 2024-03-23T23:37:50.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:37:50.171+0000 7f5284000700 1 -- 172.21.15.99:0/2597826389 wait complete. 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:51.151 DEBUG:teuthology.orchestra.run.smithi099:osd.6> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.6.service 2024-03-23T23:37:51.162 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2024-03-23T23:37:51.162 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:51.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:37:51.412 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:37:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:37:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:37:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] boot 2024-03-23T23:37:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T23:37:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:37:52.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:37:52.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:52.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] boot 2024-03-23T23:37:52.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T23:37:52.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: osd.6 [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] boot 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T23:37:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T23:37:53.254 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:53 smithi099 ceph-mon[31715]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T23:37:53.254 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:53 smithi099 ceph-mon[31715]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:53 smithi149 ceph-mon[36083]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T23:37:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:53 smithi149 ceph-mon[36083]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:53.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:53 smithi174 ceph-mon[37002]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T23:37:53.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:53 smithi174 ceph-mon[37002]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:53.442 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T23:37:53.442 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T23:37:53.443 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-23T23:37:53.443 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-23T23:37:53.443 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132059 s, 794 MB/s 2024-03-23T23:37:53.443 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-23T23:37:54.171 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-03-23T23:37:54.410 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:37:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:55 smithi149 ceph-mon[36083]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/2185710182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101450 msgr2=0x7ff4b81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/2185710182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101450 0x7ff4b81018d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009990 tx=0x7ff4ac009ca0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/2185710182 shutdown_connections 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/2185710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101e10 0x7ff4b8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/2185710182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100af0 0x7ff4b8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/2185710182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101450 0x7ff4b81018d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.103 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/2185710182 >> 172.21.15.149:0/2185710182 conn(0x7ff4b80fc690 msgr2=0x7ff4b80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/2185710182 shutdown_connections 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.100+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/2185710182 wait complete. 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 Processor -- start 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 -- start start 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100af0 0x7ff4b819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101450 0x7ff4b819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8110030 con 0x7ff4b8100af0 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b810feb0 con 0x7ff4b8101e10 2024-03-23T23:37:56.104 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4bd8e3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81101b0 con 0x7ff4b8101450 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:38962/0 (socket says 172.21.15.149:38962) 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 -- 172.21.15.149:0/4259353799 learned_addr learned my addr 172.21.15.149:0/4259353799 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b67fc700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101450 0x7ff4b819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 -- 172.21.15.149:0/4259353799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101450 msgr2=0x7ff4b819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101450 0x7ff4b819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 -- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100af0 msgr2=0x7ff4b819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100af0 0x7ff4b819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.101+0000 7ff4b77fe700 1 -- 172.21.15.149:0/4259353799 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b81a2240 con 0x7ff4b8101e10 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4b77fe700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007ee0 tx=0x7ff4b000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:37:56.105 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000dbd0 con 0x7ff4b8101e10 2024-03-23T23:37:56.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b000dd70 con 0x7ff4b8101e10 2024-03-23T23:37:56.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0005a40 con 0x7ff4b8101e10 2024-03-23T23:37:56.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81a2530 con 0x7ff4b8101e10 2024-03-23T23:37:56.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.102+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81a2af0 con 0x7ff4b8101e10 2024-03-23T23:37:56.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.103+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8068e50 con 0x7ff4b8101e10 2024-03-23T23:37:56.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.104+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff4b001a020 con 0x7ff4b8101e10 2024-03-23T23:37:56.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.104+0000 7ff4bc8e1700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 0x7ff4a00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:37:56.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.105+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7ff4b00cd3e0 con 0x7ff4b8101e10 2024-03-23T23:37:56.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.105+0000 7ff4b6ffd700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 0x7ff4a00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:37:56.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.106+0000 7ff4b6ffd700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 0x7ff4a00a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097b0 tx=0x7ff4a8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:37:56.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.109+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4b0010070 con 0x7ff4b8101e10 2024-03-23T23:37:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:55 smithi099 ceph-mon[31715]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:55 smithi174 ceph-mon[37002]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:56.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:37:56.313+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4b80625d0 con 0x7ff4a00a2890 2024-03-23T23:37:56.981 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:56.982 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:37:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:37:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:57 smithi099 ceph-mon[31715]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:57.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:57 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi099 2024-03-23T23:37:57.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:57 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi099 to 5697M 2024-03-23T23:37:57.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:57 smithi099 ceph-mon[31715]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:57.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:57 smithi149 ceph-mon[36083]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:57.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:57 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi099 2024-03-23T23:37:57.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:57 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi099 to 5697M 2024-03-23T23:37:57.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:57 smithi149 ceph-mon[36083]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:57 smithi174 ceph-mon[37002]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:37:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:57 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi099 2024-03-23T23:37:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:57 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi099 to 5697M 2024-03-23T23:37:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:57 smithi174 ceph-mon[37002]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:59.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:37:59.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/1394066560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:37:59.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:37:59.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]': finished 2024-03-23T23:37:59.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T23:37:59.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:37:59 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/1394066560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:38:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:38:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]': finished 2024-03-23T23:38:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T23:38:00.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:37:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:00.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:00.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/1394066560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:38:00.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]: dispatch 2024-03-23T23:38:00.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db44c8d6-8d15-4514-a5e3-0babe7b5b17b"}]': finished 2024-03-23T23:38:00.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T23:38:00.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:37:59 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:00 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/2229338622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:01.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:00 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/2229338622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:01.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:00 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/2229338622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:01 smithi099 ceph-mon[31715]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:02.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:01 smithi149 ceph-mon[36083]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:01 smithi174 ceph-mon[37002]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:03 smithi099 ceph-mon[31715]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:04.125 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:03 smithi149 ceph-mon[36083]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:04.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:03 smithi174 ceph-mon[37002]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:05 smithi099 ceph-mon[31715]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:05 smithi149 ceph-mon[36083]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:06.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:05 smithi174 ceph-mon[37002]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T23:38:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:07.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T23:38:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:07.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T23:38:07.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:07 smithi099 ceph-mon[31715]: Deploying daemon osd.7 on smithi149 2024-03-23T23:38:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:07 smithi099 ceph-mon[31715]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:07.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:07 smithi149 ceph-mon[36083]: Deploying daemon osd.7 on smithi149 2024-03-23T23:38:07.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:07 smithi149 ceph-mon[36083]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:08.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:07 smithi174 ceph-mon[37002]: Deploying daemon osd.7 on smithi149 2024-03-23T23:38:08.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:07 smithi174 ceph-mon[37002]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:09.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:09 smithi149 ceph-mon[36083]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:09 smithi099 ceph-mon[31715]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:10.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:09 smithi174 ceph-mon[37002]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:12.073 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:11 smithi149 ceph-mon[36083]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:11 smithi099 ceph-mon[31715]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:11 smithi174 ceph-mon[37002]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:13.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:13.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:13 smithi149 ceph-mon[36083]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:13.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:13.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:13 smithi174 ceph-mon[37002]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:13 smithi099 ceph-mon[31715]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:15.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:15 smithi149 ceph-mon[36083]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:15.953 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-03-23T23:38:15.953 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.952+0000 7ff4bc8e1700 1 -- 172.21.15.149:0/4259353799 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff4b80625d0 con 0x7ff4a00a2890 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 msgr2=0x7ff4a00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 0x7ff4a00a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097b0 tx=0x7ff4a8006d20 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 msgr2=0x7ff4b81a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007ee0 tx=0x7ff4b000d3b0 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 shutdown_connections 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4a00a2890 0x7ff4a00a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4b8101e10 0x7ff4b81a1ba0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100af0 0x7ff4b819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 --2- 172.21.15.149:0/4259353799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4b8101450 0x7ff4b819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 >> 172.21.15.149:0/4259353799 conn(0x7ff4b80fc690 msgr2=0x7ff4b80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 shutdown_connections 2024-03-23T23:38:15.955 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:38:15.955+0000 7ff4bd8e3700 1 -- 172.21.15.149:0/4259353799 wait complete. 2024-03-23T23:38:16.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:15 smithi099 ceph-mon[31715]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:15 smithi174 ceph-mon[37002]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:16.681 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.7.service 2024-03-23T23:38:16.683 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-03-23T23:38:16.684 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T23:38:16.929 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:38:17.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.051 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.051 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.051 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.051 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.051 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:16 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.188 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.189 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:16 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:17.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:16 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:18.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:17 smithi174 ceph-mon[37002]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.201 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:18.202 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T23:38:18.202 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.202 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T23:38:18.202 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:18.202 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:17 smithi149 ceph-mon[36083]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T23:38:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: from='osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T23:38:18.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:18.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:17 smithi099 ceph-mon[31715]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:38:18.564 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:38:18.358+0000 7f61070e6700 -1 osd.7 0 waiting for initial osdmap 2024-03-23T23:38:18.565 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:38:18.369+0000 7f60feed1700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112635 s, 931 MB/s 2024-03-23T23:38:18.852 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-23T23:38:19.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:18 smithi174 ceph-mon[37002]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:38:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:18 smithi174 ceph-mon[37002]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T23:38:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:18 smithi174 ceph-mon[37002]: from='osd.7 ' entity='osd.7' 2024-03-23T23:38:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:18 smithi099 ceph-mon[31715]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:38:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:18 smithi099 ceph-mon[31715]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T23:38:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:18 smithi099 ceph-mon[31715]: from='osd.7 ' entity='osd.7' 2024-03-23T23:38:19.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-mon[36083]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:38:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-mon[36083]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T23:38:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-mon[36083]: from='osd.7 ' entity='osd.7' 2024-03-23T23:38:19.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:19.556 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-03-23T23:38:19.788 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:38:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:38:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:38:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:20.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] boot 2024-03-23T23:38:20.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T23:38:20.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:20.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:19 smithi174 ceph-mon[37002]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] boot 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:19 smithi099 ceph-mon[31715]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: osd.7 [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] boot 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T23:38:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:19 smithi149 ceph-mon[36083]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-23T23:38:21.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T23:38:21.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:20 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T23:38:21.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:20 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T23:38:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:21.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:20 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 -- 172.21.15.174:0/2731452694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 msgr2=0x7fb8d010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/2731452694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d010a610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b0a0 tx=0x7fb8c800b3b0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 -- 172.21.15.174:0/2731452694 shutdown_connections 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/2731452694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d010a610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/2731452694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d01021c0 0x7fb8d01025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/2731452694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 0x7fb8d0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 -- 172.21.15.174:0/2731452694 >> 172.21.15.174:0/2731452694 conn(0x7fb8d00fd7a0 msgr2=0x7fb8d00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:38:21.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 -- 172.21.15.174:0/2731452694 shutdown_connections 2024-03-23T23:38:21.485 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.484+0000 7fb8d64b1700 1 -- 172.21.15.174:0/2731452694 wait complete. 2024-03-23T23:38:21.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 Processor -- start 2024-03-23T23:38:21.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 -- start start 2024-03-23T23:38:21.487 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d01021c0 0x7fb8d0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 0x7fb8d0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010de30 con 0x7fb8d01021c0 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010dcb0 con 0x7fb8d01033c0 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d64b1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010dfb0 con 0x7fb8d0103d60 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:36436/0 (socket says 172.21.15.174:36436) 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 -- 172.21.15.174:0/3385039114 learned_addr learned my addr 172.21.15.174:0/3385039114 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8cf7fe700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 0x7fb8d0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:21.488 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 -- 172.21.15.174:0/3385039114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 msgr2=0x7fb8d0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 0x7fb8d0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 -- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d01021c0 msgr2=0x7fb8d0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.485+0000 7fb8d4a4e700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d01021c0 0x7fb8d0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8d4a4e700 1 -- 172.21.15.174:0/3385039114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d019ffb0 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8d4a4e700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800be60 tx=0x7fb8c800be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8021020 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d01a0240 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8005520 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c801b970 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.486+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d01a07c0 con 0x7fb8d0103d60 2024-03-23T23:38:21.489 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.487+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d0068e50 con 0x7fb8d0103d60 2024-03-23T23:38:21.490 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.488+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb8c8024020 con 0x7fb8d0103d60 2024-03-23T23:38:21.490 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.488+0000 7fb8cd7fa700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:21.491 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.489+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fb8c800f020 con 0x7fb8d0103d60 2024-03-23T23:38:21.491 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.489+0000 7fb8cffff700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:21.491 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.490+0000 7fb8cffff700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 0x7fb8b80a4d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0103220 tx=0x7fb8c0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:38:21.492 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.493+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8c809cca0 con 0x7fb8d0103d60 2024-03-23T23:38:21.700 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:21.700+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d00625d0 con 0x7fb8b80a2890 2024-03-23T23:38:22.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi149 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:22.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:21 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi149 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:22.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:21 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi149 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi149 to 1935M 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:21 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:23.329 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:22 smithi174 ceph-mon[37002]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:23.329 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:22 smithi174 ceph-mon[37002]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T23:38:23.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:22 smithi099 ceph-mon[31715]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:23.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:22 smithi099 ceph-mon[31715]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T23:38:23.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:22 smithi149 ceph-mon[36083]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:23.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:22 smithi149 ceph-mon[36083]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T23:38:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:23 smithi099 ceph-mon[31715]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:23 smithi099 ceph-mon[31715]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T23:38:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:23 smithi149 ceph-mon[36083]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:23 smithi149 ceph-mon[36083]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T23:38:24.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:23 smithi174 ceph-mon[37002]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:24.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:23 smithi174 ceph-mon[37002]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T23:38:25.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/3356561112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:25 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:25 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]': finished 2024-03-23T23:38:25.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:25 smithi174 ceph-mon[37002]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T23:38:25.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:25 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:25.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/3356561112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.263 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:25 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:25 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]': finished 2024-03-23T23:38:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:25 smithi099 ceph-mon[31715]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T23:38:25.264 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:25 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/3356561112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:25 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]: dispatch 2024-03-23T23:38:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:25 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b"}]': finished 2024-03-23T23:38:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:25 smithi149 ceph-mon[36083]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T23:38:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:25 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:26 smithi099 ceph-mon[31715]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/1884542674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:26.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:26 smithi149 ceph-mon[36083]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/1884542674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:26.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:26 smithi174 ceph-mon[37002]: pgmap v205: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:26.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/1884542674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:27.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:27 smithi099 ceph-mon[31715]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:27.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:27 smithi149 ceph-mon[36083]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:27.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:27 smithi174 ceph-mon[37002]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:28.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:28 smithi099 ceph-mon[31715]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-23T23:38:28.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:28 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:38:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:28 smithi149 ceph-mon[36083]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-23T23:38:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:28 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:38:28.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:28 smithi174 ceph-mon[37002]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-23T23:38:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:28 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:38:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:29 smithi174 ceph-mon[37002]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:29.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:29 smithi099 ceph-mon[31715]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:29.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:29 smithi149 ceph-mon[36083]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:31.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:31 smithi174 ceph-mon[37002]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:31 smithi099 ceph-mon[31715]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:32.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:31 smithi149 ceph-mon[36083]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:33.045 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:32 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T23:38:33.045 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:32 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:33.045 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:32 smithi149 ceph-mon[36083]: Deploying daemon osd.8 on smithi174 2024-03-23T23:38:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:32 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T23:38:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:32 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:32 smithi099 ceph-mon[31715]: Deploying daemon osd.8 on smithi174 2024-03-23T23:38:33.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:32 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T23:38:33.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:32 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:33.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:32 smithi174 ceph-mon[37002]: Deploying daemon osd.8 on smithi174 2024-03-23T23:38:34.093 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:33 smithi174 ceph-mon[37002]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:33 smithi099 ceph-mon[31715]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:33 smithi149 ceph-mon[36083]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:35 smithi099 ceph-mon[31715]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:36.134 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:35 smithi174 ceph-mon[37002]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:36.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:35 smithi149 ceph-mon[36083]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:38.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:37 smithi174 ceph-mon[37002]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:38.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:38.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:38.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:37 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:37 smithi099 ceph-mon[31715]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:37 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:37 smithi149 ceph-mon[36083]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:37 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:40.097 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:39 smithi174 ceph-mon[37002]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:39 smithi099 ceph-mon[31715]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:40.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:39 smithi149 ceph-mon[36083]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:41.385 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-03-23T23:38:41.386 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.385+0000 7fb8cd7fa700 1 -- 172.21.15.174:0/3385039114 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb8d00625d0 con 0x7fb8b80a2890 2024-03-23T23:38:41.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 msgr2=0x7fb8b80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:41.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 0x7fb8b80a4d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0103220 tx=0x7fb8c0006d20 comp rx=0 tx=0).stop 2024-03-23T23:38:41.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 msgr2=0x7fb8d019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:41.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800be60 tx=0x7fb8c800be90 comp rx=0 tx=0).stop 2024-03-23T23:38:41.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 shutdown_connections 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb8b80a2890 0x7fb8b80a4d50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8d0103d60 0x7fb8d019f910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8d01021c0 0x7fb8d0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 --2- 172.21.15.174:0/3385039114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8d01033c0 0x7fb8d0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 >> 172.21.15.174:0/3385039114 conn(0x7fb8d00fd7a0 msgr2=0x7fb8d0106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 shutdown_connections 2024-03-23T23:38:41.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:38:41.388+0000 7fb8d64b1700 1 -- 172.21.15.174:0/3385039114 wait complete. 2024-03-23T23:38:41.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:41.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:41.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:41.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:41.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:41.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:38:41.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:42.127 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.8.service 2024-03-23T23:38:42.129 INFO:tasks.cephadm:Deploying osd.9 on smithi099 with /dev/vg_nvme/lv_1... 2024-03-23T23:38:42.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T23:38:42.571 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:38:42.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:42 smithi099 ceph-mon[31715]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T23:38:42.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:42 smithi099 ceph-mon[31715]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:42 smithi099 ceph-mon[31715]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T23:38:42.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:42.782 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:42 smithi099 ceph-mon[31715]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:42 smithi149 ceph-mon[36083]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T23:38:42.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:42 smithi149 ceph-mon[36083]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:42 smithi149 ceph-mon[36083]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T23:38:42.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:42.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:42 smithi149 ceph-mon[36083]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-mon[37002]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T23:38:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-mon[37002]: from='osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-mon[37002]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T23:38:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:42.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-mon[37002]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:38:42.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:38:42.534+0000 7f0104f63700 -1 osd.8 0 waiting for initial osdmap 2024-03-23T23:38:42.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:38:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:38:42.543+0000 7f0100d56700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:38:43.603 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:43 smithi174 ceph-mon[37002]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:38:43.603 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:43 smithi174 ceph-mon[37002]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T23:38:43.603 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:43 smithi174 ceph-mon[37002]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:43.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:43 smithi099 ceph-mon[31715]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:38:43.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:43 smithi099 ceph-mon[31715]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T23:38:43.793 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.793 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.793 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:43 smithi099 ceph-mon[31715]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:43.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:43 smithi149 ceph-mon[36083]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:38:43.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:43 smithi149 ceph-mon[36083]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T23:38:43.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:43.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:43 smithi149 ceph-mon[36083]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-23T23:38:44.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:38:44.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:38:44.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] boot 2024-03-23T23:38:44.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T23:38:44.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:44.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:44 smithi149 ceph-mon[36083]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] boot 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:44 smithi174 ceph-mon[37002]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T23:38:44.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:38:44.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:38:44.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: osd.8 [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] boot 2024-03-23T23:38:44.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T23:38:44.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T23:38:44.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:44 smithi099 ceph-mon[31715]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.036715 s, 286 MB/s 2024-03-23T23:38:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.758 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:45.759 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:45.759 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.759 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:45 smithi099 ceph-mon[31715]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T23:38:45.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:45.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:45.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:45.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.896 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:45 smithi149 ceph-mon[36083]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T23:38:45.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:45.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:38:45.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:38:45.957 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:45 smithi174 ceph-mon[37002]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T23:38:46.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-03-23T23:38:46.860 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:46 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi174 2024-03-23T23:38:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:46 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi174 to 5693M 2024-03-23T23:38:46.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:46 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi174 2024-03-23T23:38:46.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:46 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi174 to 5693M 2024-03-23T23:38:46.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:46 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi174 2024-03-23T23:38:46.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:46 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi174 to 5693M 2024-03-23T23:38:46.985 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:38:47.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:47 smithi099 ceph-mon[31715]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T23:38:47.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:47 smithi099 ceph-mon[31715]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:47 smithi149 ceph-mon[36083]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T23:38:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:47 smithi149 ceph-mon[36083]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:47 smithi174 ceph-mon[37002]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T23:38:47.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:47 smithi174 ceph-mon[37002]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:48.835 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:48 smithi099 ceph-mon[31715]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T23:38:48.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:48 smithi149 ceph-mon[36083]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T23:38:48.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:48 smithi174 ceph-mon[37002]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T23:38:49.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 -- 172.21.15.99:0/2987972368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718102c50 msgr2=0x7fb718105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:49.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 --2- 172.21.15.99:0/2987972368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718102c50 0x7fb718105040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009990 tx=0x7fb70c009ca0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 -- 172.21.15.99:0/2987972368 shutdown_connections 2024-03-23T23:38:49.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 --2- 172.21.15.99:0/2987972368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718107eb0 0x7fb71810a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 --2- 172.21.15.99:0/2987972368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718102c50 0x7fb718105040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 --2- 172.21.15.99:0/2987972368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb718107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 -- 172.21.15.99:0/2987972368 >> 172.21.15.99:0/2987972368 conn(0x7fb7180fc660 msgr2=0x7fb7180feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:38:49.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 -- 172.21.15.99:0/2987972368 shutdown_connections 2024-03-23T23:38:49.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.364+0000 7fb71f600700 1 -- 172.21.15.99:0/2987972368 wait complete. 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 Processor -- start 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 -- start start 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 0x7fb71819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb71819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718107eb0 0x7fb7181a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:49.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810de20 con 0x7fb718107eb0 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810dca0 con 0x7fb718105580 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71f600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71810dfa0 con 0x7fb718102c50 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.366+0000 7fb71d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718107eb0 0x7fb7181a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb717e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb71819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb717e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb71819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:60394/0 (socket says 172.21.15.99:60394) 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 0x7fb71819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb717e00700 1 -- 172.21.15.99:0/3512392236 learned_addr learned my addr 172.21.15.99:0/3512392236 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:38:49.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 -- 172.21.15.99:0/3512392236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 msgr2=0x7fb71819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb71819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 -- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718107eb0 msgr2=0x7fb7181a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718107eb0 0x7fb7181a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 -- 172.21.15.99:0/3512392236 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7181a2280 con 0x7fb718102c50 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.367+0000 7fb71cc00700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 0x7fb71819a620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00b7b0 tx=0x7fb70c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.368+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c026070 con 0x7fb718102c50 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.368+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb70c003d70 con 0x7fb718102c50 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.368+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c021410 con 0x7fb718102c50 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.368+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7181a2510 con 0x7fb718102c50 2024-03-23T23:38:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.368+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7181a2960 con 0x7fb718102c50 2024-03-23T23:38:49.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.369+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb718068e50 con 0x7fb718102c50 2024-03-23T23:38:49.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.371+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb70c029070 con 0x7fb718102c50 2024-03-23T23:38:49.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.372+0000 7fb715600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 0x7fb7000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:38:49.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.373+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fb70c0cf080 con 0x7fb718102c50 2024-03-23T23:38:49.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.373+0000 7fb717e00700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 0x7fb7000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:38:49.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.373+0000 7fb717e00700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 0x7fb7000a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb704005950 tx=0x7fb70400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:38:49.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.375+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb70c013070 con 0x7fb718102c50 2024-03-23T23:38:49.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:38:49.614+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7180625d0 con 0x7fb7000a2890 2024-03-23T23:38:49.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:49 smithi099 ceph-mon[31715]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:49.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:49 smithi149 ceph-mon[36083]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:49.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:49 smithi174 ceph-mon[37002]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:38:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:50 smithi099 ceph-mon[31715]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:50.893 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:50 smithi149 ceph-mon[36083]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:50.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:50.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:50.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:50.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:50 smithi174 ceph-mon[37002]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:38:50.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:38:50.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:38:50.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:38:51.804 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:51 smithi099 ceph-mon[31715]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:38:51.897 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:51 smithi149 ceph-mon[36083]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:38:52.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:51 smithi174 ceph-mon[37002]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T23:38:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:53 smithi099 ceph-mon[31715]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-23T23:38:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]: dispatch 2024-03-23T23:38:54.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]': finished 2024-03-23T23:38:54.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:53 smithi099 ceph-mon[31715]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T23:38:54.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:38:54.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:53 smithi149 ceph-mon[36083]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-23T23:38:54.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]: dispatch 2024-03-23T23:38:54.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]': finished 2024-03-23T23:38:54.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:53 smithi149 ceph-mon[36083]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T23:38:54.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:38:54.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:53 smithi174 ceph-mon[37002]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-23T23:38:54.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]: dispatch 2024-03-23T23:38:54.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2947235888' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3720287f-1023-43eb-bce3-280e52715f41"}]': finished 2024-03-23T23:38:54.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:53 smithi174 ceph-mon[37002]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T23:38:54.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:38:55.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:54 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/5240632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:55.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:54 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/5240632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:54 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/5240632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:38:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:55 smithi099 ceph-mon[31715]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:38:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:55 smithi149 ceph-mon[36083]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:38:56.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:55 smithi174 ceph-mon[37002]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:38:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:57 smithi099 ceph-mon[31715]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:38:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:57 smithi149 ceph-mon[36083]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:38:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:57 smithi174 ceph-mon[37002]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:38:59.960 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:38:59 smithi099 ceph-mon[31715]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:39:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:38:59 smithi149 ceph-mon[36083]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:39:00.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:38:59 smithi174 ceph-mon[37002]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T23:39:01.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:01 smithi174 ceph-mon[37002]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:02.045 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:01 smithi099 ceph-mon[31715]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:02.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:01 smithi149 ceph-mon[36083]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:04.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:03 smithi099 ceph-mon[31715]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:04.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T23:39:04.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:04.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:03 smithi099 ceph-mon[31715]: Deploying daemon osd.9 on smithi099 2024-03-23T23:39:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:03 smithi149 ceph-mon[36083]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T23:39:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:03 smithi149 ceph-mon[36083]: Deploying daemon osd.9 on smithi099 2024-03-23T23:39:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:03 smithi174 ceph-mon[37002]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T23:39:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:03 smithi174 ceph-mon[37002]: Deploying daemon osd.9 on smithi099 2024-03-23T23:39:05.961 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:05 smithi099 ceph-mon[31715]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:05 smithi149 ceph-mon[36083]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:05 smithi174 ceph-mon[37002]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:07.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:07 smithi099 ceph-mon[31715]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:07 smithi149 ceph-mon[36083]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:08.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:07 smithi174 ceph-mon[37002]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:09 smithi099 ceph-mon[31715]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:10.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:09 smithi149 ceph-mon[36083]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:10.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:09 smithi174 ceph-mon[37002]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:11 smithi099 ceph-mon[31715]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:11 smithi149 ceph-mon[36083]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:11 smithi174 ceph-mon[37002]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:13 smithi099 ceph-mon[31715]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:13 smithi149 ceph-mon[36083]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:14.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:13 smithi174 ceph-mon[37002]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:15 smithi099 ceph-mon[31715]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:15 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:15 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:15 smithi099 ceph-mon[31715]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T23:39:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:15 smithi149 ceph-mon[36083]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:15 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:15 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:15 smithi149 ceph-mon[36083]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T23:39:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:15 smithi174 ceph-mon[37002]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:15 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:15 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:15 smithi174 ceph-mon[37002]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T23:39:16.872 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 9 on host 'smithi099' 2024-03-23T23:39:16.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.870+0000 7fb715600700 1 -- 172.21.15.99:0/3512392236 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb7180625d0 con 0x7fb7000a2890 2024-03-23T23:39:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.874+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 msgr2=0x7fb7000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.874+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 0x7fb7000a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb704005950 tx=0x7fb70400b500 comp rx=0 tx=0).stop 2024-03-23T23:39:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 msgr2=0x7fb71819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 0x7fb71819a620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00b7b0 tx=0x7fb70c00bbb0 comp rx=0 tx=0).stop 2024-03-23T23:39:16.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 shutdown_connections 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7000a2890 0x7fb7000a4d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb718107eb0 0x7fb7181a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb718102c50 0x7fb71819a620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 --2- 172.21.15.99:0/3512392236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb718105580 0x7fb71819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 >> 172.21.15.99:0/3512392236 conn(0x7fb7180fc660 msgr2=0x7fb7181024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 shutdown_connections 2024-03-23T23:39:16.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:39:16.875+0000 7fb71f600700 1 -- 172.21.15.99:0/3512392236 wait complete. 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:17 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:17.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:17 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-23T23:39:17.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:17.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:17 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:17.757 DEBUG:teuthology.orchestra.run.smithi099:osd.9> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.9.service 2024-03-23T23:39:17.763 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2024-03-23T23:39:17.763 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T23:39:18.011 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:39:18.392 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:18 smithi149 ceph-mon[36083]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:39:18.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:18 smithi149 ceph-mon[36083]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T23:39:18.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:18.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:18 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:18.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:18 smithi099 ceph-mon[31715]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:39:18.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:18 smithi099 ceph-mon[31715]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T23:39:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:18.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:18 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:18 smithi174 ceph-mon[37002]: from='osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-23T23:39:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:18 smithi174 ceph-mon[37002]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T23:39:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:18.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:18 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:39:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:39:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] boot 2024-03-23T23:39:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T23:39:19.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:19.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:19 smithi149 ceph-mon[36083]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:19.551 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:39:19.551 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:39:19.551 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] boot 2024-03-23T23:39:19.552 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T23:39:19.552 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:19.552 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:19 smithi099 ceph-mon[31715]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: osd.9 [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] boot 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T23:39:19.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:19 smithi174 ceph-mon[37002]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118788 s, 883 MB/s 2024-03-23T23:39:20.072 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-23T23:39:20.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:20 smithi149 ceph-mon[36083]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T23:39:20.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:20 smithi174 ceph-mon[37002]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T23:39:20.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-03-23T23:39:20.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:20 smithi099 ceph-mon[31715]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T23:39:21.009 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:39:21.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:21 smithi149 ceph-mon[36083]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:21.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:21 smithi174 ceph-mon[37002]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:21.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:21 smithi099 ceph-mon[31715]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- 172.21.15.149:0/2388107273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 msgr2=0x7fc4b8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 --2- 172.21.15.149:0/2388107273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8103840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80099c0 tx=0x7fc4a8009cd0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- 172.21.15.149:0/2388107273 shutdown_connections 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 --2- 172.21.15.149:0/2388107273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b8103d80 0x7fc4b810a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 --2- 172.21.15.149:0/2388107273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 0x7fc4b8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 --2- 172.21.15.149:0/2388107273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8103840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- 172.21.15.149:0/2388107273 >> 172.21.15.149:0/2388107273 conn(0x7fc4b80fd7a0 msgr2=0x7fc4b80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- 172.21.15.149:0/2388107273 shutdown_connections 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- 172.21.15.149:0/2388107273 wait complete. 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 Processor -- start 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.713+0000 7fc4c0210700 1 -- start start 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 0x7fc4b8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b8103d80 0x7fc4b819f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b810de50 con 0x7fc4b8103d80 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b810dcd0 con 0x7fc4b81033e0 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b810dfd0 con 0x7fc4b81021e0 2024-03-23T23:39:22.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:37462/0 (socket says 172.21.15.149:37462) 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 -- 172.21.15.149:0/1971036185 learned_addr learned my addr 172.21.15.149:0/1971036185 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bdfac700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 0x7fc4b8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 -- 172.21.15.149:0/1971036185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 msgr2=0x7fc4b8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 0x7fc4b8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 -- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b8103d80 msgr2=0x7fc4b819f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b8103d80 0x7fc4b819f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 -- 172.21.15.149:0/1971036185 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b81a0090 con 0x7fc4b81033e0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4bd7ab700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8003890 tx=0x7fc4a80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8027070 con 0x7fc4b81033e0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b81a0320 con 0x7fc4b81033e0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4a80041d0 con 0x7fc4b81033e0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.714+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800f460 con 0x7fc4b81033e0 2024-03-23T23:39:22.717 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.715+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b81a0850 con 0x7fc4b81033e0 2024-03-23T23:39:22.718 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.715+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b8068e50 con 0x7fc4b81033e0 2024-03-23T23:39:22.718 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.716+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc4a802a070 con 0x7fc4b81033e0 2024-03-23T23:39:22.718 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.717+0000 7fc4aeffd700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 0x7fc4a40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:22.718 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.717+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc4a80cf0c0 con 0x7fc4b81033e0 2024-03-23T23:39:22.719 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.717+0000 7fc4bdfac700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 0x7fc4a40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:22.719 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.718+0000 7fc4bdfac700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 0x7fc4a40a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4005950 tx=0x7fc4b400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:39:22.721 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.722+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4a8013070 con 0x7fc4b81033e0 2024-03-23T23:39:22.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:22.923+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b80625d0 con 0x7fc4a40a2910 2024-03-23T23:39:23.853 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi099 2024-03-23T23:39:23.853 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.853 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi099 to 4272M 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:23.854 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:23.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi099 2024-03-23T23:39:23.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi099 to 4272M 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:23.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:23.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:23.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:23.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi099 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi099 to 4272M 2024-03-23T23:39:23.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:23.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:23.925 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:23.925 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:26.022 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:25 smithi099 ceph-mon[31715]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:26.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:25 smithi149 ceph-mon[36083]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:26.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:25 smithi174 ceph-mon[37002]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/1965121656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]': finished 2024-03-23T23:39:27.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T23:39:27.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:27.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.149:0/3667071217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/1965121656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]': finished 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:27.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.149:0/3667071217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/1965121656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]: dispatch 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b134a1cf-77b1-4218-b117-38efbd289712"}]': finished 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:27.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.149:0/3667071217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:27 smithi099 ceph-mon[31715]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:27 smithi149 ceph-mon[36083]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:27 smithi174 ceph-mon[37002]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:29 smithi099 ceph-mon[31715]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:29 smithi149 ceph-mon[36083]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:30.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:29 smithi174 ceph-mon[37002]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:32.084 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:31 smithi149 ceph-mon[36083]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:32.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:31 smithi099 ceph-mon[31715]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:32.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:31 smithi174 ceph-mon[37002]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:33 smithi099 ceph-mon[31715]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:34.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T23:39:34.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:33 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:34.123 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:33 smithi149 ceph-mon[36083]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:34.123 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T23:39:34.123 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:33 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:33 smithi174 ceph-mon[37002]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T23:39:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:33 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:35.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:34 smithi099 ceph-mon[31715]: Deploying daemon osd.10 on smithi149 2024-03-23T23:39:35.141 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:34 smithi149 ceph-mon[36083]: Deploying daemon osd.10 on smithi149 2024-03-23T23:39:35.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:34 smithi174 ceph-mon[37002]: Deploying daemon osd.10 on smithi149 2024-03-23T23:39:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:35 smithi099 ceph-mon[31715]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:36.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:35 smithi149 ceph-mon[36083]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:35 smithi174 ceph-mon[37002]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:37 smithi099 ceph-mon[31715]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:38.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:37 smithi149 ceph-mon[36083]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:38.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:37 smithi174 ceph-mon[37002]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:39 smithi149 ceph-mon[36083]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:39 smithi099 ceph-mon[31715]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:39 smithi174 ceph-mon[37002]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:41 smithi099 ceph-mon[31715]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:42.137 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:41 smithi149 ceph-mon[36083]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:42.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:41 smithi174 ceph-mon[37002]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:43.312 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2024-03-23T23:39:43.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.312+0000 7fc4aeffd700 1 -- 172.21.15.149:0/1971036185 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc4b80625d0 con 0x7fc4a40a2910 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 msgr2=0x7fc4a40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 0x7fc4a40a4dd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4005950 tx=0x7fc4b400a300 comp rx=0 tx=0).stop 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 msgr2=0x7fc4b8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8003890 tx=0x7fc4a80047b0 comp rx=0 tx=0).stop 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 shutdown_connections 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4a40a2910 0x7fc4a40a4dd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4b8103d80 0x7fc4b819f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:43.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4b81021e0 0x7fc4b8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:43.316 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 --2- 172.21.15.149:0/1971036185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4b81033e0 0x7fc4b8198970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:43.316 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 >> 172.21.15.149:0/1971036185 conn(0x7fc4b80fd7a0 msgr2=0x7fc4b81066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:39:43.316 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 shutdown_connections 2024-03-23T23:39:43.316 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-23T23:39:43.315+0000 7fc4c0210700 1 -- 172.21.15.149:0/1971036185 wait complete. 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:43.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:43.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T23:39:43.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:43.924 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:39:44.039 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.10.service 2024-03-23T23:39:44.042 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-03-23T23:39:44.042 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T23:39:44.288 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:39:44.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-mon[36083]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T23:39:44.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-mon[36083]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:44.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-mon[36083]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T23:39:44.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:44.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-mon[36083]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:44 smithi174 ceph-mon[37002]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T23:39:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:44 smithi174 ceph-mon[37002]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:44 smithi174 ceph-mon[37002]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T23:39:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:44.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:44 smithi174 ceph-mon[37002]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:44.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:44 smithi099 ceph-mon[31715]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T23:39:44.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:44 smithi099 ceph-mon[31715]: from='osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:44.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:44 smithi099 ceph-mon[31715]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T23:39:44.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:44.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:44 smithi099 ceph-mon[31715]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-23T23:39:45.305 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:39:44.916+0000 7f97b175c700 -1 osd.10 0 waiting for initial osdmap 2024-03-23T23:39:45.305 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:39:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:39:44.930+0000 7f97a9547700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:45.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:45 smithi149 ceph-mon[36083]: from='osd.10 ' entity='osd.10' 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:45.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:45 smithi174 ceph-mon[37002]: from='osd.10 ' entity='osd.10' 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-23T23:39:45.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:45 smithi099 ceph-mon[31715]: from='osd.10 ' entity='osd.10' 2024-03-23T23:39:46.361 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T23:39:46.361 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T23:39:46.361 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-23T23:39:46.362 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-23T23:39:46.362 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111824 s, 938 MB/s 2024-03-23T23:39:46.362 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-23T23:39:46.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:46 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:39:46.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:46 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:39:46.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:46 smithi174 ceph-mon[37002]: osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] boot 2024-03-23T23:39:46.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:46 smithi174 ceph-mon[37002]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T23:39:46.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:46.829 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:46 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:39:46.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:46 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:39:46.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:46 smithi149 ceph-mon[36083]: osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] boot 2024-03-23T23:39:46.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:46 smithi149 ceph-mon[36083]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T23:39:46.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:46 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:39:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:46 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:39:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:46 smithi099 ceph-mon[31715]: osd.10 [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] boot 2024-03-23T23:39:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:46 smithi099 ceph-mon[31715]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T23:39:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T23:39:47.073 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-03-23T23:39:47.316 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: pgmap v263: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:47 smithi174 ceph-mon[37002]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T23:39:47.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: pgmap v263: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:47.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:47.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:47 smithi099 ceph-mon[31715]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T23:39:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T23:39:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: pgmap v263: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:39:47.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:47 smithi149 ceph-mon[36083]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T23:39:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:48 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi149 2024-03-23T23:39:48.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:48 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-23T23:39:48.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:48 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi149 2024-03-23T23:39:48.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:48 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-23T23:39:48.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:48 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi149 2024-03-23T23:39:48.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:48 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi149 to 1451M 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/1241082478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b04100bc0 msgr2=0x7f1b0410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/1241082478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b04100bc0 0x7f1b0410a790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b0a0 tx=0x7f1afc00b3b0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/1241082478 shutdown_connections 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/1241082478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b04100bc0 0x7f1b0410a790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/1241082478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b041037a0 0x7f1b04103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/1241082478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100220 0x7f1b04100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.960 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/1241082478 >> 172.21.15.174:0/1241082478 conn(0x7f1b040fbe00 msgr2=0x7f1b040fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/1241082478 shutdown_connections 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.958+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/1241082478 wait complete. 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 Processor -- start 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 -- start start 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100220 0x7f1b041983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b04100bc0 0x7f1b04198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0410dde0 con 0x7f1b04100220 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0410dc60 con 0x7f1b04100bc0 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0c0a2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0410df60 con 0x7f1b041037a0 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:48.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43392/0 (socket says 172.21.15.174:43392) 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 -- 172.21.15.174:0/319093733 learned_addr learned my addr 172.21.15.174:0/319093733 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0963d700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b04100bc0 0x7f1b04198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 -- 172.21.15.174:0/319093733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b04100bc0 msgr2=0x7f1b04198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b04100bc0 0x7f1b04198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 -- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100220 msgr2=0x7f1b041983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100220 0x7f1b041983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.959+0000 7f1b0a63f700 1 -- 172.21.15.174:0/319093733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b041a0020 con 0x7f1b041037a0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1b0a63f700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009cd0 tx=0x7f1afc007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc021070 con 0x7f1b041037a0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1afc01db30 con 0x7f1b041037a0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005680 con 0x7f1b041037a0 2024-03-23T23:39:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b041a02b0 con 0x7f1b041037a0 2024-03-23T23:39:48.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.960+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041a07e0 con 0x7f1b041037a0 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.961+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04068e50 con 0x7f1b041037a0 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.962+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1afc024070 con 0x7f1b041037a0 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.962+0000 7f1afaffd700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 0x7f1af00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.962+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f1afc00f020 con 0x7f1b041037a0 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.963+0000 7f1b09e3e700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 0x7f1af00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:39:48.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.964+0000 7f1b09e3e700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 0x7f1af00a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b04069b60 tx=0x7f1af4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:39:48.966 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:48.967+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1afc02b020 con 0x7f1b041037a0 2024-03-23T23:39:49.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:39:49.169+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b040625d0 con 0x7f1af00a2890 2024-03-23T23:39:49.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:49 smithi174 ceph-mon[37002]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T23:39:49.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:49 smithi174 ceph-mon[37002]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:49.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:49.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:49.807 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:49.867 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:49 smithi099 ceph-mon[31715]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T23:39:49.867 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:49 smithi099 ceph-mon[31715]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:49.867 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:49.867 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:49.867 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:49.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:49 smithi149 ceph-mon[36083]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T23:39:49.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:49 smithi149 ceph-mon[36083]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:39:49.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T23:39:49.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T23:39:49.902 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:39:50.817 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:50 smithi174 ceph-mon[37002]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:50.817 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:50 smithi174 ceph-mon[37002]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T23:39:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:50 smithi099 ceph-mon[31715]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:50 smithi099 ceph-mon[31715]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T23:39:50.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:50 smithi149 ceph-mon[36083]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:39:50.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:50 smithi149 ceph-mon[36083]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T23:39:51.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:51 smithi099 ceph-mon[31715]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-23T23:39:51.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:51 smithi149 ceph-mon[36083]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-23T23:39:51.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:51 smithi174 ceph-mon[37002]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-23T23:39:52.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:52 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/20077053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:52 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:52 smithi099 ceph-mon[31715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]': finished 2024-03-23T23:39:52.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:52 smithi099 ceph-mon[31715]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T23:39:52.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:39:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:52 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/20077053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:52 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:52 smithi149 ceph-mon[36083]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]': finished 2024-03-23T23:39:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:52 smithi149 ceph-mon[36083]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T23:39:52.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:39:52.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:52 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/20077053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:52 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]: dispatch 2024-03-23T23:39:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:52 smithi174 ceph-mon[37002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1fa75b5-9238-4a2e-bce2-4ab94569fbfd"}]': finished 2024-03-23T23:39:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:52 smithi174 ceph-mon[37002]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T23:39:52.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:39:53.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.174:0/3531440462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:53.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:53 smithi099 ceph-mon[31715]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-23T23:39:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.174:0/3531440462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:53 smithi149 ceph-mon[36083]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-23T23:39:53.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.174:0/3531440462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T23:39:53.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:53 smithi174 ceph-mon[37002]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-23T23:39:55.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:55 smithi174 ceph-mon[37002]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-23T23:39:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:55 smithi099 ceph-mon[31715]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-23T23:39:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:55 smithi149 ceph-mon[36083]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-23T23:39:58.039 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:57 smithi174 ceph-mon[37002]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:39:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:57 smithi099 ceph-mon[31715]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:39:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:57 smithi149 ceph-mon[36083]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T23:40:00.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:59 smithi174 ceph-mon[37002]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:40:00.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:59 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T23:40:00.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:39:59 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:59 smithi099 ceph-mon[31715]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:40:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T23:40:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:39:59 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:59 smithi149 ceph-mon[36083]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-23T23:40:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:59 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T23:40:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:39:59 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:00.911 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:00 smithi174 ceph-mon[37002]: Deploying daemon osd.11 on smithi174 2024-03-23T23:40:00.911 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:00 smithi174 ceph-mon[37002]: overall HEALTH_OK 2024-03-23T23:40:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:00 smithi099 ceph-mon[31715]: Deploying daemon osd.11 on smithi174 2024-03-23T23:40:01.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:00 smithi099 ceph-mon[31715]: overall HEALTH_OK 2024-03-23T23:40:01.146 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:00 smithi149 ceph-mon[36083]: Deploying daemon osd.11 on smithi174 2024-03-23T23:40:01.147 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:00 smithi149 ceph-mon[36083]: overall HEALTH_OK 2024-03-23T23:40:02.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:01 smithi099 ceph-mon[31715]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:02.114 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:01 smithi174 ceph-mon[37002]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:02.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:01 smithi149 ceph-mon[36083]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:03 smithi099 ceph-mon[31715]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:03 smithi149 ceph-mon[36083]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:03 smithi174 ceph-mon[37002]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:06.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:05 smithi174 ceph-mon[37002]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:06.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:05 smithi099 ceph-mon[31715]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:06.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:05 smithi149 ceph-mon[36083]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:08.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:07 smithi099 ceph-mon[31715]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:07 smithi149 ceph-mon[36083]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:08.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:07 smithi174 ceph-mon[37002]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:09.063 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-03-23T23:40:09.064 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.063+0000 7f1afaffd700 1 -- 172.21.15.174:0/319093733 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1b040625d0 con 0x7f1af00a2890 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 msgr2=0x7f1af00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 0x7f1af00a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b04069b60 tx=0x7f1af4006d20 comp rx=0 tx=0).stop 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 msgr2=0x7f1b0419f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009cd0 tx=0x7f1afc007820 comp rx=0 tx=0).stop 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 shutdown_connections 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1af00a2890 0x7f1af00a4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:09.065 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b041037a0 0x7f1b0419f980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:09.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b04100220 0x7f1b041983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:09.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 --2- 172.21.15.174:0/319093733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b04100bc0 0x7f1b04198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:09.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 >> 172.21.15.174:0/319093733 conn(0x7f1b040fbe00 msgr2=0x7f1b041067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:09.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 shutdown_connections 2024-03-23T23:40:09.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:09.065+0000 7f1b0c0a2700 1 -- 172.21.15.174:0/319093733 wait complete. 2024-03-23T23:40:09.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:09.707 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.11.service 2024-03-23T23:40:09.710 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-23T23:40:09.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T23:40:10.165 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:40:10.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:10 smithi099 ceph-mon[31715]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T23:40:10.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:10 smithi099 ceph-mon[31715]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:10 smithi099 ceph-mon[31715]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T23:40:10.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:10.558 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:10 smithi099 ceph-mon[31715]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.613 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-mon[37002]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T23:40:10.613 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-mon[37002]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.613 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-mon[37002]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T23:40:10.613 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:10.613 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-mon[37002]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:10 smithi149 ceph-mon[36083]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T23:40:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:10 smithi149 ceph-mon[36083]: from='osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:10 smithi149 ceph-mon[36083]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T23:40:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:10 smithi149 ceph-mon[36083]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-23T23:40:10.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:40:10.687+0000 7fcc05170700 -1 osd.11 0 waiting for initial osdmap 2024-03-23T23:40:10.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:40:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:40:10.701+0000 7fcbfcf5b700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T23:40:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:40:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T23:40:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: from='osd.11 ' entity='osd.11' 2024-03-23T23:40:11.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:11 smithi099 ceph-mon[31715]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: from='osd.11 ' entity='osd.11' 2024-03-23T23:40:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:11 smithi149 ceph-mon[36083]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: from='osd.11 ' entity='osd.11' 2024-03-23T23:40:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:11 smithi174 ceph-mon[37002]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-23T23:40:12.284 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: purged_snaps scrub starts 2024-03-23T23:40:12.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: purged_snaps scrub ok 2024-03-23T23:40:12.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] boot 2024-03-23T23:40:12.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T23:40:12.285 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:12 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.353 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: purged_snaps scrub starts 2024-03-23T23:40:12.353 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: purged_snaps scrub ok 2024-03-23T23:40:12.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] boot 2024-03-23T23:40:12.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T23:40:12.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:12 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 -- 172.21.15.99:0/1489750707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa288103d70 msgr2=0x7fa28810a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 --2- 172.21.15.99:0/1489750707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa288103d70 0x7fa28810a620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa28000b0d0 tx=0x7fa28000b3e0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 -- 172.21.15.99:0/1489750707 shutdown_connections 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 --2- 172.21.15.99:0/1489750707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa288103d70 0x7fa28810a620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 --2- 172.21.15.99:0/1489750707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2881021d0 0x7fa2881025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 --2- 172.21.15.99:0/1489750707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.557+0000 7fa28e400700 1 -- 172.21.15.99:0/1489750707 >> 172.21.15.99:0/1489750707 conn(0x7fa2880fd730 msgr2=0x7fa2880ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.558+0000 7fa28e400700 1 -- 172.21.15.99:0/1489750707 shutdown_connections 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.558+0000 7fa28e400700 1 -- 172.21.15.99:0/1489750707 wait complete. 2024-03-23T23:40:12.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 Processor -- start 2024-03-23T23:40:12.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 -- start start 2024-03-23T23:40:12.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2881021d0 0x7fa288075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:12.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:12.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288103d70 0x7fa288074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:12.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28810ddb0 con 0x7fa2881021d0 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28810dc30 con 0x7fa288103d70 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.559+0000 7fa28e400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28810df30 con 0x7fa2881033d0 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:54564/0 (socket says 172.21.15.99:54564) 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 -- 172.21.15.99:0/1425448727 learned_addr learned my addr 172.21.15.99:0/1425448727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa287400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2881021d0 0x7fa288075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa287e00700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288103d70 0x7fa288074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 -- 172.21.15.99:0/1425448727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288103d70 msgr2=0x7fa288074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288103d70 0x7fa288074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 -- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2881021d0 msgr2=0x7fa288075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2881021d0 0x7fa288075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 -- 172.21.15.99:0/1425448727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288075370 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.560+0000 7fa286a00700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa27400eb30 tx=0x7fa27400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.561+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400cbd0 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.561+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa274004500 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.561+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2881a24c0 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.561+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274010430 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.561+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2881a29d0 con 0x7fa2881033d0 2024-03-23T23:40:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.562+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa288068e50 con 0x7fa2881033d0 2024-03-23T23:40:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.564+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa274003f00 con 0x7fa2881033d0 2024-03-23T23:40:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.565+0000 7fa27fe00700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 0x7fa26c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.565+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2740ce990 con 0x7fa2881033d0 2024-03-23T23:40:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.565+0000 7fa287400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 0x7fa26c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.566+0000 7fa287400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 0x7fa26c0a4d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa278009680 tx=0x7fa278009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:12.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.569+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa27409be30 con 0x7fa2881033d0 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: purged_snaps scrub starts 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: purged_snaps scrub ok 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: osd.11 [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] boot 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T23:40:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:12 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T23:40:12.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.797+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa28802de60 con 0x7fa2881033d0 2024-03-23T23:40:12.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.797+0000 7fa27fe00700 1 -- 172.21.15.99:0/1425448727 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa27409b730 con 0x7fa2881033d0 2024-03-23T23:40:12.798 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:12.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 msgr2=0x7fa26c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:12.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 0x7fa26c0a4d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa278009680 tx=0x7fa278009260 comp rx=0 tx=0).stop 2024-03-23T23:40:12.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 msgr2=0x7fa288074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:12.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa27400eb30 tx=0x7fa27400ee40 comp rx=0 tx=0).stop 2024-03-23T23:40:12.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 shutdown_connections 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa26c0a2890 0x7fa26c0a4d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288103d70 0x7fa288074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2881021d0 0x7fa288075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 --2- 172.21.15.99:0/1425448727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2881033d0 0x7fa288074580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.800+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 >> 172.21.15.99:0/1425448727 conn(0x7fa2880fd730 msgr2=0x7fa288106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.801+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 shutdown_connections 2024-03-23T23:40:12.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:12.801+0000 7fa28e400700 1 -- 172.21.15.99:0/1425448727 wait complete. 2024-03-23T23:40:13.651 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711237211,"num_in_osds":12,"osd_in_since":1711237191,"num_remapped_pgs":0} 2024-03-23T23:40:13.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1425448727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: Detected new or changed devices on smithi174 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:13.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: Adjusting osd_memory_target on smithi174 to 4270M 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:13.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:13 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.061 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1425448727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: Detected new or changed devices on smithi174 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: Adjusting osd_memory_target on smithi174 to 4270M 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:14.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:13 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1425448727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: Detected new or changed devices on smithi174 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: Adjusting osd_memory_target on smithi174 to 4270M 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:14.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:13 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:15.937 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:15 smithi099 ceph-mon[31715]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:15 smithi149 ceph-mon[36083]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:15 smithi174 ceph-mon[37002]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:16.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 -- 172.21.15.99:0/2629024053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca01062b0 msgr2=0x7fbca010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2629024053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca01062b0 0x7fbca010de80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009f90 tx=0x7fbc9800b330 comp rx=0 tx=0).stop 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 -- 172.21.15.99:0/2629024053 shutdown_connections 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2629024053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca01062b0 0x7fbca010de80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2629024053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 0x7fbca01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2629024053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca0101b80 0x7fbca0105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.242+0000 7fbc9f200700 1 -- 172.21.15.99:0/2629024053 >> 172.21.15.99:0/2629024053 conn(0x7fbca00fca60 msgr2=0x7fbca00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.243+0000 7fbc9f200700 1 -- 172.21.15.99:0/2629024053 shutdown_connections 2024-03-23T23:40:16.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.243+0000 7fbc9f200700 1 -- 172.21.15.99:0/2629024053 wait complete. 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 Processor -- start 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 -- start start 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 0x7fbca01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 0x7fbca019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca01123f0 con 0x7fbca01062b0 2024-03-23T23:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0112270 con 0x7fbca0101b80 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.244+0000 7fbc9f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0112570 con 0x7fbca01011d0 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc9e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 0x7fbca019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc9de00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 0x7fbca01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39176/0 (socket says 172.21.15.99:39176) 2024-03-23T23:40:16.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 -- 172.21.15.99:0/2821575697 learned_addr learned my addr 172.21.15.99:0/2821575697 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:16.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 -- 172.21.15.99:0/2821575697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 msgr2=0x7fbca01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:16.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 0x7fbca01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 -- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 msgr2=0x7fbca019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:16.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 0x7fbca019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 -- 172.21.15.99:0/2821575697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca019afd0 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc9e800700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 0x7fbca019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.245+0000 7fbc97e00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800ec70 tx=0x7fbc8800ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.246+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800cc90 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.246+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca019b2c0 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.246+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc8800ce30 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.246+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88010430 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.246+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca01a7580 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.247+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca0068e50 con 0x7fbca0101b80 2024-03-23T23:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.249+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbc88004000 con 0x7fbca0101b80 2024-03-23T23:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.250+0000 7fbc96a00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 0x7fbc840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.250+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc880ce8f0 con 0x7fbca0101b80 2024-03-23T23:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.250+0000 7fbc9de00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 0x7fbc840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.251+0000 7fbc9de00700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 0x7fbc840a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbca019b9b0 tx=0x7fbc90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:16.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.252+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbc8809bd90 con 0x7fbca0101b80 2024-03-23T23:40:16.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.477+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca019bda0 con 0x7fbca0101b80 2024-03-23T23:40:16.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.478+0000 7fbc96a00700 1 -- 172.21.15.99:0/2821575697 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fbc8809b690 con 0x7fbca0101b80 2024-03-23T23:40:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:16.485 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":81,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:40:12.704666+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 msgr2=0x7fbc840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 0x7fbc840a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbca019b9b0 tx=0x7fbc90009260 comp rx=0 tx=0).stop 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 msgr2=0x7fbca01a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800ec70 tx=0x7fbc8800ef80 comp rx=0 tx=0).stop 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 shutdown_connections 2024-03-23T23:40:16.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbc840a2890 0x7fbc840a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbca01062b0 0x7fbca019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca01011d0 0x7fbca01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 --2- 172.21.15.99:0/2821575697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca0101b80 0x7fbca01a0bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.481+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 >> 172.21.15.99:0/2821575697 conn(0x7fbca00fca60 msgr2=0x7fbca010c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.482+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 shutdown_connections 2024-03-23T23:40:16.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:16.482+0000 7fbc9f200700 1 -- 172.21.15.99:0/2821575697 wait complete. 2024-03-23T23:40:17.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:16 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2821575697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:17.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:16 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2821575697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:17.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:16 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2821575697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:17.354 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:40:17.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-23T23:40:17.793 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:40:17.985 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:17 smithi099 ceph-mon[31715]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:17 smithi149 ceph-mon[36083]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:17 smithi174 ceph-mon[37002]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:19.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:19 smithi099 ceph-mon[31715]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.084+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3004163146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee0102410 msgr2=0x7f0ee01027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3004163146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee0102410 0x7f0ee01027f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400b0a0 tx=0x7f0ed400b3b0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3004163146 shutdown_connections 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3004163146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee01074f0 0x7f0ee010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3004163146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee0102410 0x7f0ee01027f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3004163146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102dc0 0x7f0ee0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3004163146 >> 172.21.15.99:0/3004163146 conn(0x7f0ee00fdc60 msgr2=0x7f0ee0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:20.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3004163146 shutdown_connections 2024-03-23T23:40:20.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.085+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3004163146 wait complete. 2024-03-23T23:40:20.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 Processor -- start 2024-03-23T23:40:20.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 -- start start 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102410 0x7f0ee019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 0x7f0ee019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 0x7f0ee01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0112540 con 0x7f0ee0102dc0 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee01123c0 con 0x7f0ee01074f0 2024-03-23T23:40:20.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.086+0000 7f0ee7e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee01126c0 con 0x7f0ee0102410 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 0x7f0ee019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 0x7f0ee01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 0x7f0ee019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47034/0 (socket says 172.21.15.99:47034) 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee4a00700 1 -- 172.21.15.99:0/3897536943 learned_addr learned my addr 172.21.15.99:0/3897536943 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5400700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102410 0x7f0ee019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 -- 172.21.15.99:0/3897536943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102410 msgr2=0x7f0ee019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102410 0x7f0ee019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 -- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 msgr2=0x7f0ee019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:20.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 0x7f0ee019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 -- 172.21.15.99:0/3897536943 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0198f80 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.087+0000 7f0ee5e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 0x7f0ee01987a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e970 tx=0x7f0ed800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.088+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8004220 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.088+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee0199270 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.088+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed8004d10 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.088+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8010700 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.088+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee01a5480 con 0x7f0ee01074f0 2024-03-23T23:40:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.089+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee01038f0 con 0x7f0ee01074f0 2024-03-23T23:40:20.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.091+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0ed8004750 con 0x7f0ee01074f0 2024-03-23T23:40:20.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.092+0000 7f0ede000700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 0x7f0ec80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:20.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.093+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ed80ce320 con 0x7f0ee01074f0 2024-03-23T23:40:20.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.093+0000 7f0ee5400700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 0x7f0ec80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:20.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.093+0000 7f0ee5400700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 0x7f0ec80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009e60 tx=0x7f0ed400b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.095+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ed809b7c0 con 0x7f0ee01074f0 2024-03-23T23:40:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:19 smithi149 ceph-mon[36083]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:19 smithi174 ceph-mon[37002]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:20.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.315+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ee002da30 con 0x7f0ee01074f0 2024-03-23T23:40:20.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.316+0000 7f0ede000700 1 -- 172.21.15.99:0/3897536943 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ed809b0c0 con 0x7f0ee01074f0 2024-03-23T23:40:20.316 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:40:20.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 msgr2=0x7f0ec80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:20.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 0x7f0ec80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009e60 tx=0x7f0ed400b740 comp rx=0 tx=0).stop 2024-03-23T23:40:20.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 msgr2=0x7f0ee01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:20.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 0x7f0ee01987a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e970 tx=0x7f0ed800ec80 comp rx=0 tx=0).stop 2024-03-23T23:40:20.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 shutdown_connections 2024-03-23T23:40:20.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0ec80a2890 0x7f0ec80a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ee01074f0 0x7f0ee01987a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ee0102410 0x7f0ee019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 --2- 172.21.15.99:0/3897536943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ee0102dc0 0x7f0ee019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:20.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 >> 172.21.15.99:0/3897536943 conn(0x7f0ee00fdc60 msgr2=0x7f0ee00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:20.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 shutdown_connections 2024-03-23T23:40:20.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:20.319+0000 7f0ee7e00700 1 -- 172.21.15.99:0/3897536943 wait complete. 2024-03-23T23:40:21.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:20 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3897536943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:21.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:20 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3897536943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:21.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:20 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3897536943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:21.207 INFO:tasks.cephadm:Adding mds.a on smithi099 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.d on smithi099 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.g on smithi099 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.j on smithi099 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.b on smithi149 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.e on smithi149 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.h on smithi149 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.k on smithi149 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-03-23T23:40:21.208 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-03-23T23:40:21.208 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-03-23T23:40:21.457 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:40:21.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:21 smithi174 ceph-mon[37002]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:22.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:21 smithi099 ceph-mon[31715]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:21 smithi149 ceph-mon[36083]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.027+0000 7ff51b4fb700 1 -- 172.21.15.174:0/1422002847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514102310 msgr2=0x7ff514102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.027+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/1422002847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514102310 0x7ff514102770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009ca0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 -- 172.21.15.174:0/1422002847 shutdown_connections 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/1422002847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514102cb0 0x7ff514109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/1422002847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff514101110 0x7ff514101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/1422002847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514102310 0x7ff514102770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 -- 172.21.15.174:0/1422002847 >> 172.21.15.174:0/1422002847 conn(0x7ff5140fc6b0 msgr2=0x7ff5140feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 -- 172.21.15.174:0/1422002847 shutdown_connections 2024-03-23T23:40:23.028 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 -- 172.21.15.174:0/1422002847 wait complete. 2024-03-23T23:40:23.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 Processor -- start 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.028+0000 7ff51b4fb700 1 -- start start 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff514102310 0x7ff514197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514102cb0 0x7ff51419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51410cdf0 con 0x7ff514102310 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51410cc70 con 0x7ff514102cb0 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51410cf70 con 0x7ff514101110 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43710/0 (socket says 172.21.15.174:43710) 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 -- 172.21.15.174:0/3991053045 learned_addr learned my addr 172.21.15.174:0/3991053045 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 -- 172.21.15.174:0/3991053045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514102cb0 msgr2=0x7ff51419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514102cb0 0x7ff51419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.031 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 -- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff514102310 msgr2=0x7ff514197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff514102310 0x7ff514197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 -- 172.21.15.174:0/3991053045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51419f0a0 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff519297700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff51000b530 tx=0x7ff51000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000f8a0 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff510010470 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51406a8d0 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510019660 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.029+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51406aee0 con 0x7ff514101110 2024-03-23T23:40:23.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.030+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff514068e50 con 0x7ff514101110 2024-03-23T23:40:23.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.031+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff510010620 con 0x7ff514101110 2024-03-23T23:40:23.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.032+0000 7ff50a7fc700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 0x7ff5000a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:23.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.032+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5100cfdd0 con 0x7ff514101110 2024-03-23T23:40:23.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.032+0000 7ff518a96700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 0x7ff5000a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:23.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.033+0000 7ff518a96700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 0x7ff5000a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff504005f50 tx=0x7ff504005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:23.036 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.036+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff510015030 con 0x7ff514101110 2024-03-23T23:40:23.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.241+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) v1 -- 0x7ff51406b560 con 0x7ff5000a2910 2024-03-23T23:40:23.258 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.258+0000 7ff50a7fc700 1 -- 172.21.15.174:0/3991053045 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff51406b560 con 0x7ff5000a2910 2024-03-23T23:40:23.258 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 msgr2=0x7ff5000a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 0x7ff5000a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff504005f50 tx=0x7ff504005e80 comp rx=0 tx=0).stop 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 msgr2=0x7ff514197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff51000b530 tx=0x7ff51000b8f0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 shutdown_connections 2024-03-23T23:40:23.260 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff5000a2910 0x7ff5000a4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514102cb0 0x7ff51419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff514101110 0x7ff514197440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 --2- 172.21.15.174:0/3991053045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff514102310 0x7ff514197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 >> 172.21.15.174:0/3991053045 conn(0x7ff5140fc6b0 msgr2=0x7ff514105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 shutdown_connections 2024-03-23T23:40:23.261 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-23T23:40:23.260+0000 7ff51b4fb700 1 -- 172.21.15.174:0/3991053045 wait complete. 2024-03-23T23:40:23.874 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:23 smithi174 ceph-mon[37002]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:23.874 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:23.874 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:23 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:23.875 DEBUG:teuthology.orchestra.run.smithi099:mds.a> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.a.service 2024-03-23T23:40:23.879 DEBUG:teuthology.orchestra.run.smithi099:mds.d> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.d.service 2024-03-23T23:40:23.926 DEBUG:teuthology.orchestra.run.smithi099:mds.g> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.g.service 2024-03-23T23:40:23.971 DEBUG:teuthology.orchestra.run.smithi099:mds.j> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.j.service 2024-03-23T23:40:24.015 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.b.service 2024-03-23T23:40:24.017 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.e.service 2024-03-23T23:40:24.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:23 smithi099 ceph-mon[31715]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:24.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:24.023 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:23 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:24.041 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:23 smithi149 ceph-mon[36083]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:24.041 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:24.041 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:23 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:40:24.042 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.h.service 2024-03-23T23:40:24.084 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.k.service 2024-03-23T23:40:24.127 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.c.service 2024-03-23T23:40:24.130 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.f.service 2024-03-23T23:40:24.173 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.i.service 2024-03-23T23:40:24.215 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.l.service 2024-03-23T23:40:24.260 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-23T23:40:24.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 -- 172.21.15.99:0/110461837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9098104ad0 msgr2=0x7f9098104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 --2- 172.21.15.99:0/110461837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9098104ad0 0x7f9098104f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9080009900 tx=0x7f908002bc30 comp rx=0 tx=0).stop 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 -- 172.21.15.99:0/110461837 shutdown_connections 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 --2- 172.21.15.99:0/110461837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 0x7f909810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 --2- 172.21.15.99:0/110461837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9098104ad0 0x7f9098104f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.471+0000 7f909f800640 1 --2- 172.21.15.99:0/110461837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90981038e0 0x7f9098103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.472+0000 7f909f800640 1 -- 172.21.15.99:0/110461837 >> 172.21.15.99:0/110461837 conn(0x7f90980fee90 msgr2=0x7f90981012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.472+0000 7f909f800640 1 -- 172.21.15.99:0/110461837 shutdown_connections 2024-03-23T23:40:24.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.472+0000 7f909f800640 1 -- 172.21.15.99:0/110461837 wait complete. 2024-03-23T23:40:24.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 Processor -- start 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 -- start start 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 0x7f9098199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 0x7f90981a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909810f560 con 0x7f9098104ad0 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909810f3e0 con 0x7f9098105490 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909f800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909810f6e0 con 0x7f90981038e0 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f908fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f909ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 0x7f9098199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f908fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47052/0 (socket says 172.21.15.99:47052) 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.473+0000 7f908fe00640 1 -- 172.21.15.99:0/388769541 learned_addr learned my addr 172.21.15.99:0/388769541 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909d800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 0x7f90981a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 -- 172.21.15.99:0/388769541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 msgr2=0x7f90981a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 0x7f90981a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 -- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 msgr2=0x7f909819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 -- 172.21.15.99:0/388769541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90981a17d0 con 0x7f90981038e0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f908fe00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909ce00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 0x7f9098199bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9c0 tx=0x7f908800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800ccc0 con 0x7f90981038e0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90880035f0 con 0x7f90981038e0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90981a1ac0 con 0x7f90981038e0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.474+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800ce60 con 0x7f90981038e0 2024-03-23T23:40:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.475+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90981a1fe0 con 0x7f90981038e0 2024-03-23T23:40:24.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.476+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909810ac50 con 0x7f90981038e0 2024-03-23T23:40:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.478+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f90880037a0 con 0x7f90981038e0 2024-03-23T23:40:24.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.479+0000 7f9096000640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 0x7f907c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.480+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90880cea00 con 0x7f90981038e0 2024-03-23T23:40:24.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.480+0000 7f908fe00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 0x7f907c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.480+0000 7f908fe00640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 0x7f907c0a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f908002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.485+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f908809bea0 con 0x7f90981038e0 2024-03-23T23:40:24.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.683+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90980622a0 con 0x7f90981038e0 2024-03-23T23:40:24.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.685+0000 7f9096000640 1 -- 172.21.15.99:0/388769541 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7f908809b7a0 con 0x7f90981038e0 2024-03-23T23:40:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:24.686 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":81,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:40:12.704666+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:40:24.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 msgr2=0x7f907c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 0x7f907c0a4cc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9080000c00 tx=0x7f908002c880 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 msgr2=0x7f9098199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 0x7f9098199bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f908800e9c0 tx=0x7f908800ee80 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 shutdown_connections 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.688+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f907c0a2800 0x7f907c0a4cc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9098105490 0x7f90981a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098104ad0 0x7f909819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 --2- 172.21.15.99:0/388769541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90981038e0 0x7f9098199bb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 >> 172.21.15.99:0/388769541 conn(0x7f90980fee90 msgr2=0x7f9098107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 shutdown_connections 2024-03-23T23:40:24.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.689+0000 7f909f800640 1 -- 172.21.15.99:0/388769541 wait complete. 2024-03-23T23:40:24.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:40:24.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:40:24.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 -- 172.21.15.99:0/3563673832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 msgr2=0x7fdf5010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 --2- 172.21.15.99:0/3563673832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf5010bd20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009ca0 tx=0x7fdf4802c100 comp rx=0 tx=0).stop 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 -- 172.21.15.99:0/3563673832 shutdown_connections 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 --2- 172.21.15.99:0/3563673832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf5010bd20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 --2- 172.21.15.99:0/3563673832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50104ad0 0x7fdf50104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 --2- 172.21.15.99:0/3563673832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf501038e0 0x7fdf50103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 -- 172.21.15.99:0/3563673832 >> 172.21.15.99:0/3563673832 conn(0x7fdf500fee50 msgr2=0x7fdf50101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 -- 172.21.15.99:0/3563673832 shutdown_connections 2024-03-23T23:40:24.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.918+0000 7fdf57200640 1 -- 172.21.15.99:0/3563673832 wait complete. 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.919+0000 7fdf57200640 1 Processor -- start 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.919+0000 7fdf57200640 1 -- start start 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf501038e0 0x7fdf50199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5010f560 con 0x7fdf50105490 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5010f3e0 con 0x7fdf501038e0 2024-03-23T23:40:24.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf57200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5010f6e0 con 0x7fdf50104ad0 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf47400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf47400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35378/0 (socket says 172.21.15.99:35378) 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf47400640 1 -- 172.21.15.99:0/3253777183 learned_addr learned my addr 172.21.15.99:0/3253777183 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47068/0 (socket says 172.21.15.99:47068) 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf4fe00640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf501038e0 0x7fdf50199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 -- 172.21.15.99:0/3253777183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 msgr2=0x7fdf5019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 -- 172.21.15.99:0/3253777183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf501038e0 msgr2=0x7fdf50199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf501038e0 0x7fdf50199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf55200640 1 -- 172.21.15.99:0/3253777183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf501a17c0 con 0x7fdf50105490 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.920+0000 7fdf47400640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf55200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf4802c640 tx=0x7fdf48035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:24.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48039040 con 0x7fdf50105490 2024-03-23T23:40:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf48044070 con 0x7fdf50105490 2024-03-23T23:40:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf501a1a50 con 0x7fdf50105490 2024-03-23T23:40:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480044a0 con 0x7fdf50105490 2024-03-23T23:40:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.921+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501a1f60 con 0x7fdf50105490 2024-03-23T23:40:24.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.922+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5010ac00 con 0x7fdf50105490 2024-03-23T23:40:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.924+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdf48004640 con 0x7fdf50105490 2024-03-23T23:40:24.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.925+0000 7fdf4d600640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:24.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.926+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf480f1720 con 0x7fdf50105490 2024-03-23T23:40:24.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.926+0000 7fdf4fe00640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:24.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.927+0000 7fdf4fe00640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 0x7fdf340a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005fd0 tx=0x7fdf4000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:24.929+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf480bebc0 con 0x7fdf50105490 2024-03-23T23:40:25.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:40:25.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-03-23T23:40:25.030 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.031 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:24 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/388769541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:40:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-03-23T23:40:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:25.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:25.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:25.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:25.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:24 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/388769541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.127+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf5002d070 con 0x7fdf50105490 2024-03-23T23:40:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.128+0000 7fdf4d600640 1 -- 172.21.15.99:0/3253777183 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf480098b0 con 0x7fdf50105490 2024-03-23T23:40:25.128 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:40:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 msgr2=0x7fdf340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 0x7fdf340a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005fd0 tx=0x7fdf4000b390 comp rx=0 tx=0).stop 2024-03-23T23:40:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 msgr2=0x7fdf501a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf4802c640 tx=0x7fdf48035d90 comp rx=0 tx=0).stop 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 shutdown_connections 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf50105490 0x7fdf501a1150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 --2- 172.21.15.99:0/3253777183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf501038e0 0x7fdf50199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 >> 172.21.15.99:0/3253777183 conn(0x7fdf500fee50 msgr2=0x7fdf50107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 shutdown_connections 2024-03-23T23:40:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.131+0000 7fdf57200640 1 -- 172.21.15.99:0/3253777183 wait complete. 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:24 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/388769541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:40:25.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 -- 172.21.15.99:0/2721852927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464109890 msgr2=0x7ff464110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 --2- 172.21.15.99:0/2721852927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464109890 0x7ff464110120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009ca0 tx=0x7ff45c02c100 comp rx=0 tx=0).stop 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 -- 172.21.15.99:0/2721852927 shutdown_connections 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 --2- 172.21.15.99:0/2721852927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464109890 0x7ff464110120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 --2- 172.21.15.99:0/2721852927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 0x7ff464109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 --2- 172.21.15.99:0/2721852927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464107ce0 0x7ff464108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 -- 172.21.15.99:0/2721852927 >> 172.21.15.99:0/2721852927 conn(0x7ff464076cb0 msgr2=0x7ff4640790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.357+0000 7ff46ac00640 1 -- 172.21.15.99:0/2721852927 shutdown_connections 2024-03-23T23:40:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.358+0000 7ff46ac00640 1 -- 172.21.15.99:0/2721852927 wait complete. 2024-03-23T23:40:25.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.358+0000 7ff46ac00640 1 Processor -- start 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 -- start start 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 0x7ff46419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 0x7ff46419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 0x7ff4641a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff464113940 con 0x7ff464108ed0 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4641137c0 con 0x7ff464107ce0 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff46ac00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff464113ac0 con 0x7ff464109890 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff45b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 0x7ff46419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff463e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 0x7ff46419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff463e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 0x7ff46419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34220/0 (socket says 172.21.15.99:34220) 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff468c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 0x7ff4641a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:25.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.359+0000 7ff463e00640 1 -- 172.21.15.99:0/1158029324 learned_addr learned my addr 172.21.15.99:0/1158029324 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 -- 172.21.15.99:0/1158029324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 msgr2=0x7ff46419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 0x7ff46419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 -- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 msgr2=0x7ff46419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 0x7ff46419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 -- 172.21.15.99:0/1158029324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4641a5b70 con 0x7ff464109890 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff468c00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 0x7ff4641a5500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02c640 tx=0x7ff45c035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c039040 con 0x7ff464109890 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff45c044070 con 0x7ff464109890 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4641a5e00 con 0x7ff464109890 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0044a0 con 0x7ff464109890 2024-03-23T23:40:25.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.360+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4641a6260 con 0x7ff464109890 2024-03-23T23:40:25.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.361+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46410f000 con 0x7ff464109890 2024-03-23T23:40:25.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.363+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff45c004640 con 0x7ff464109890 2024-03-23T23:40:25.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.364+0000 7ff461600640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 0x7ff4480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:25.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.365+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff45c0f0d50 con 0x7ff464109890 2024-03-23T23:40:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.365+0000 7ff463e00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 0x7ff4480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.365+0000 7ff463e00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 0x7ff4480a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff45400bc80 tx=0x7ff45400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.369+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff45c0be1f0 con 0x7ff464109890 2024-03-23T23:40:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.601+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff464062340 con 0x7ff464109890 2024-03-23T23:40:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.602+0000 7ff461600640 1 -- 172.21.15.99:0/1158029324 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7ff45c0098b0 con 0x7ff464109890 2024-03-23T23:40:25.602 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:25.603 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-23T23:40:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 1 2024-03-23T23:40:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.605+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 msgr2=0x7ff4480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.605+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 0x7ff4480a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff45400bc80 tx=0x7ff45400b410 comp rx=0 tx=0).stop 2024-03-23T23:40:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.605+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 msgr2=0x7ff4641a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.605+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 0x7ff4641a5500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff45c02c640 tx=0x7ff45c035d90 comp rx=0 tx=0).stop 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 shutdown_connections 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff4480a2800 0x7ff4480a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff464109890 0x7ff4641a5500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff464108ed0 0x7ff46419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 --2- 172.21.15.99:0/1158029324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff464107ce0 0x7ff46419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 >> 172.21.15.99:0/1158029324 conn(0x7ff464076cb0 msgr2=0x7ff46410c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 shutdown_connections 2024-03-23T23:40:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:25.606+0000 7ff46ac00640 1 -- 172.21.15.99:0/1158029324 wait complete. 2024-03-23T23:40:26.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:25 smithi174 ceph-mon[37002]: Deploying daemon mds.l on smithi174 2024-03-23T23:40:26.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:25 smithi174 ceph-mon[37002]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:26.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3253777183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:26.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1158029324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:25 smithi099 ceph-mon[31715]: Deploying daemon mds.l on smithi174 2024-03-23T23:40:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:25 smithi099 ceph-mon[31715]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3253777183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:26.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1158029324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:25 smithi149 ceph-mon[36083]: Deploying daemon mds.l on smithi174 2024-03-23T23:40:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:25 smithi149 ceph-mon[36083]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3253777183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:40:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1158029324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:26.673 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:26 smithi174 systemd[1]: Starting Ceph mds.l for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:26.984 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:26 smithi174 podman[67176]: 2024-03-23 23:40:26.610958089 +0000 UTC m=+0.017395148 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:26.984 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:26 smithi174 podman[67176]: 2024-03-23 23:40:26.732574457 +0000 UTC m=+0.139011545 container create b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-23T23:40:26.984 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:26 smithi174 podman[67176]: 2024-03-23 23:40:26.985324037 +0000 UTC m=+0.391761082 container init b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-23T23:40:27.423 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:26 smithi174 podman[67176]: 2024-03-23 23:40:26.987828093 +0000 UTC m=+0.394265141 container start b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T23:40:27.423 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mds[67195]: -- 172.21.15.174:0/3417568117 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562f1d645a40 con 0x562f1d618800 2024-03-23T23:40:27.424 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: starting mds.l at 2024-03-23T23:40:27.424 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:27 smithi174 bash[67176]: b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf 2024-03-23T23:40:27.424 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:40:27 smithi174 systemd[1]: Started Ceph mds.l for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:27.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:27.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 -- 172.21.15.99:0/1188516199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1052a0 msgr2=0x7fc3dc10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/1188516199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc10bb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d402c100 comp rx=0 tx=0).stop 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 -- 172.21.15.99:0/1188516199 shutdown_connections 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/1188516199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc10bb30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/1188516199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1048e0 0x7fc3dc104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/1188516199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 -- 172.21.15.99:0/1188516199 >> 172.21.15.99:0/1188516199 conn(0x7fc3dc0fec80 msgr2=0x7fc3dc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 -- 172.21.15.99:0/1188516199 shutdown_connections 2024-03-23T23:40:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.816+0000 7fc3dbe00640 1 -- 172.21.15.99:0/1188516199 wait complete. 2024-03-23T23:40:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.817+0000 7fc3dbe00640 1 Processor -- start 2024-03-23T23:40:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.817+0000 7fc3dbe00640 1 -- start start 2024-03-23T23:40:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.817+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.817+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1048e0 0x7fc3dc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.817+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc10f370 con 0x7fc3dc1048e0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc10f1f0 con 0x7fc3dc1036f0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc10f4f0 con 0x7fc3dc1052a0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34240/0 (socket says 172.21.15.99:34240) 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 -- 172.21.15.99:0/3788673408 learned_addr learned my addr 172.21.15.99:0/3788673408 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3da000640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1048e0 0x7fc3dc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3db400640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 -- 172.21.15.99:0/3788673408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1052a0 msgr2=0x7fc3dc1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc1a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 -- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1048e0 msgr2=0x7fc3dc199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1048e0 0x7fc3dc199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 -- 172.21.15.99:0/3788673408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc1a15a0 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.818+0000 7fc3daa00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e9c0 tx=0x7fc3c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.819+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400cd80 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.819+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3c40035f0 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.819+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc1a1890 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.819+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4010690 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.819+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc1a1e30 con 0x7fc3dc1036f0 2024-03-23T23:40:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.820+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc10a9b0 con 0x7fc3dc1036f0 2024-03-23T23:40:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.821+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc3c40037a0 con 0x7fc3dc1036f0 2024-03-23T23:40:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.823+0000 7fc3d3400640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 0x7fc3c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.823+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3c40cea40 con 0x7fc3dc1036f0 2024-03-23T23:40:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.823+0000 7fc3da000640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 0x7fc3c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.824+0000 7fc3da000640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 0x7fc3c00a4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc19ae90 tx=0x7fc3cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:27.827+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc3c409bee0 con 0x7fc3dc1036f0 2024-03-23T23:40:28.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.055+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3dc0622a0 con 0x7fc3dc1036f0 2024-03-23T23:40:28.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.056+0000 7fc3d3400640 1 -- 172.21.15.99:0/3788673408 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fc3c409b7e0 con 0x7fc3dc1036f0 2024-03-23T23:40:28.056 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:28.056 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:28.056 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2024-03-23T23:40:28.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 msgr2=0x7fc3c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 0x7fc3c00a4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc19ae90 tx=0x7fc3cc00b390 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 msgr2=0x7fc3dc199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400e9c0 tx=0x7fc3c400ee80 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 shutdown_connections 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc3c00a2880 0x7fc3c00a4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3dc1052a0 0x7fc3dc1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3dc1048e0 0x7fc3dc199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 --2- 172.21.15.99:0/3788673408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3dc1036f0 0x7fc3dc199980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.058+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 >> 172.21.15.99:0/3788673408 conn(0x7fc3dc0fec80 msgr2=0x7fc3dc107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.059+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 shutdown_connections 2024-03-23T23:40:28.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:28.059+0000 7fc3dbe00640 1 -- 172.21.15.99:0/3788673408 wait complete. 2024-03-23T23:40:28.074 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:28.074 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6832/4085463188,v1:172.21.15.174:6833/4085463188] up:boot 2024-03-23T23:40:28.074 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: fsmap 1 up:standby 2024-03-23T23:40:28.074 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:40:28.074 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:28.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:28.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:27 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6832/4085463188,v1:172.21.15.174:6833/4085463188] up:boot 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: fsmap 1 up:standby 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:28.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:28.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:27 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6832/4085463188,v1:172.21.15.174:6833/4085463188] up:boot 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: fsmap 1 up:standby 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:27 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:28 smithi149 ceph-mon[36083]: Deploying daemon mds.k on smithi149 2024-03-23T23:40:28.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3788673408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:28 smithi099 ceph-mon[31715]: Deploying daemon mds.k on smithi149 2024-03-23T23:40:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3788673408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:28 smithi174 ceph-mon[37002]: Deploying daemon mds.k on smithi149 2024-03-23T23:40:29.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3788673408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:29.644 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 systemd[1]: Starting Ceph mds.k for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:29.943 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:29 smithi149 ceph-mon[36083]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:29.944 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 podman[65236]: 2024-03-23 23:40:29.592272426 +0000 UTC m=+0.017897082 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:29.944 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 podman[65236]: 2024-03-23 23:40:29.707861266 +0000 UTC m=+0.133485904 container create e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T23:40:29.944 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 podman[65236]: 2024-03-23 23:40:29.944384647 +0000 UTC m=+0.370009290 container init e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-23T23:40:30.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:29 smithi099 ceph-mon[31715]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:30.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:29 smithi174 ceph-mon[37002]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1773731383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1028a0 msgr2=0x7fe2dc102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1773731383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc102d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009860 tx=0x7fe2c802bb70 comp rx=0 tx=0).stop 2024-03-23T23:40:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1773731383 shutdown_connections 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1773731383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc103260 0x7fe2dc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1773731383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc102d20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1773731383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1773731383 >> 172.21.15.99:0/1773731383 conn(0x7fe2dc0fcc80 msgr2=0x7fe2dc0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1773731383 shutdown_connections 2024-03-23T23:40:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.285+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1773731383 wait complete. 2024-03-23T23:40:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.286+0000 7fe2e1a00640 1 Processor -- start 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.286+0000 7fe2e1a00640 1 -- start start 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc10d330 con 0x7fe2dc103260 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc10d1b0 con 0x7fe2dc1016b0 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2e1a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc10d4b0 con 0x7fe2dc1028a0 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2db400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2db400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58522/0 (socket says 172.21.15.99:58522) 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2db400640 1 -- 172.21.15.99:0/1348504289 learned_addr learned my addr 172.21.15.99:0/1348504289 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2daa00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.287+0000 7fe2da000640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 -- 172.21.15.99:0/1348504289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc1028a0 msgr2=0x7fe2dc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 -- 172.21.15.99:0/1348504289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 msgr2=0x7fe2dc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 -- 172.21.15.99:0/1348504289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc19f5c0 con 0x7fe2dc103260 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2daa00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:30.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2db400640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400eb10 tx=0x7fe2d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4004b20 con 0x7fe2dc103260 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc19f8b0 con 0x7fe2dc103260 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2d4004cc0 con 0x7fe2dc103260 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4010760 con 0x7fe2dc103260 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.288+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc19fe20 con 0x7fe2dc103260 2024-03-23T23:40:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.290+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc108a50 con 0x7fe2dc103260 2024-03-23T23:40:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.292+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe2d4005020 con 0x7fe2dc103260 2024-03-23T23:40:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.294+0000 7fe2d3400640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 0x7fe2c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.294+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2d40ced40 con 0x7fe2dc103260 2024-03-23T23:40:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.294+0000 7fe2daa00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 0x7fe2c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.295+0000 7fe2daa00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 0x7fe2c00a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0096c0 tx=0x7fe2cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:30.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.298+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2d409c1e0 con 0x7fe2dc103260 2024-03-23T23:40:30.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 podman[65236]: 2024-03-23 23:40:29.947174002 +0000 UTC m=+0.372798649 container start e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-23T23:40:30.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 ceph-mds[65255]: -- 172.21.15.149:0/3893792789 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e69043a40 con 0x556e69016800 2024-03-23T23:40:30.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: starting mds.k at 2024-03-23T23:40:30.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:30 smithi149 bash[65236]: e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 2024-03-23T23:40:30.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:40:30 smithi149 systemd[1]: Started Ceph mds.k for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:30.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.541+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2dc0622a0 con 0x7fe2dc103260 2024-03-23T23:40:30.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.542+0000 7fe2d3400640 1 -- 172.21.15.99:0/1348504289 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe2d409bae0 con 0x7fe2dc103260 2024-03-23T23:40:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:30.542 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:30.542 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.544+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 msgr2=0x7fe2c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.544+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 0x7fe2c00a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0096c0 tx=0x7fe2cc009260 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 msgr2=0x7fe2dc19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400eb10 tx=0x7fe2d400c2d0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 shutdown_connections 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2c00a2800 0x7fe2c00a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc103260 0x7fe2dc19ef50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2dc1028a0 0x7fe2dc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 --2- 172.21.15.99:0/1348504289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2dc1016b0 0x7fe2dc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 >> 172.21.15.99:0/1348504289 conn(0x7fe2dc0fcc80 msgr2=0x7fe2dc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:30.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 shutdown_connections 2024-03-23T23:40:30.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:30.545+0000 7fe2e1a00640 1 -- 172.21.15.99:0/1348504289 wait complete. 2024-03-23T23:40:31.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:31.289 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: Deploying daemon mds.j on smithi099 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1348504289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6832/2919310963,v1:172.21.15.149:6833/2919310963] up:boot 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: fsmap 2 up:standby 2024-03-23T23:40:31.290 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:31 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: Deploying daemon mds.j on smithi099 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1348504289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6832/2919310963,v1:172.21.15.149:6833/2919310963] up:boot 2024-03-23T23:40:31.339 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: fsmap 2 up:standby 2024-03-23T23:40:31.340 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:31 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: Deploying daemon mds.j on smithi099 2024-03-23T23:40:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1348504289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:31.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:31.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6832/2919310963,v1:172.21.15.149:6833/2919310963] up:boot 2024-03-23T23:40:31.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: fsmap 2 up:standby 2024-03-23T23:40:31.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:31 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:40:32.563 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:32.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 -- 172.21.15.99:0/2579160319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f0102140 msgr2=0x7fe2f0104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:32.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 --2- 172.21.15.99:0/2579160319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f0102140 0x7fe2f0104530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402bc30 comp rx=0 tx=0).stop 2024-03-23T23:40:32.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 -- 172.21.15.99:0/2579160319 shutdown_connections 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 --2- 172.21.15.99:0/2579160319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f01073a0 0x7fe2f01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 --2- 172.21.15.99:0/2579160319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0104a70 0x7fe2f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 --2- 172.21.15.99:0/2579160319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f0102140 0x7fe2f0104530 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 -- 172.21.15.99:0/2579160319 >> 172.21.15.99:0/2579160319 conn(0x7fe2f00fbb30 msgr2=0x7fe2f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 -- 172.21.15.99:0/2579160319 shutdown_connections 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.855+0000 7fe2f8400640 1 -- 172.21.15.99:0/2579160319 wait complete. 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.856+0000 7fe2f8400640 1 Processor -- start 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.856+0000 7fe2f8400640 1 -- start start 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0104a70 0x7fe2f01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f01073a0 0x7fe2f0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f010d390 con 0x7fe2f01073a0 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f010d210 con 0x7fe2f0104a70 2024-03-23T23:40:32.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f8400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f010d510 con 0x7fe2f0102140 2024-03-23T23:40:32.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:32.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50948/0 (socket says 172.21.15.99:50948) 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f5a00640 1 -- 172.21.15.99:0/1155298489 learned_addr learned my addr 172.21.15.99:0/1155298489 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f5000640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0104a70 0x7fe2f01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.857+0000 7fe2f6400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f01073a0 0x7fe2f0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.858+0000 7fe2f5a00640 1 -- 172.21.15.99:0/1155298489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0104a70 msgr2=0x7fe2f01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.858+0000 7fe2f5a00640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0104a70 0x7fe2f01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.858+0000 7fe2f5a00640 1 -- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f01073a0 msgr2=0x7fe2f0100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.858+0000 7fe2f5a00640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f01073a0 0x7fe2f0100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.858+0000 7fe2f5a00640 1 -- 172.21.15.99:0/1155298489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0100ec0 con 0x7fe2f0102140 2024-03-23T23:40:32.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2f5a00640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000c00 tx=0x7fe2e402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:32.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4043020 con 0x7fe2f0102140 2024-03-23T23:40:32.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2e40054e0 con 0x7fe2f0102140 2024-03-23T23:40:32.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4003c40 con 0x7fe2f0102140 2024-03-23T23:40:32.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f0071d30 con 0x7fe2f0102140 2024-03-23T23:40:32.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.859+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f00722d0 con 0x7fe2f0102140 2024-03-23T23:40:32.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.861+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0068490 con 0x7fe2f0102140 2024-03-23T23:40:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.864+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe2e4046020 con 0x7fe2f0102140 2024-03-23T23:40:32.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.865+0000 7fe2ee000640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 0x7fe2d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:32.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.866+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2e40f0660 con 0x7fe2f0102140 2024-03-23T23:40:32.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.866+0000 7fe2f5000640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 0x7fe2d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:32.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.867+0000 7fe2f5000640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 0x7fe2d80a4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0101820 tx=0x7fe2dc018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:32.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:32.870+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2e404d020 con 0x7fe2f0102140 2024-03-23T23:40:33.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.117+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2f00fd2c0 con 0x7fe2f0102140 2024-03-23T23:40:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.117+0000 7fe2ee000640 1 -- 172.21.15.99:0/1155298489 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fe2e40bdba0 con 0x7fe2f0102140 2024-03-23T23:40:33.118 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:33.118 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2024-03-23T23:40:33.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 msgr2=0x7fe2d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:33.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 0x7fe2d80a4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0101820 tx=0x7fe2dc018040 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 msgr2=0x7fe2f0101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4000c00 tx=0x7fe2e402c650 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 shutdown_connections 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2d80a2880 0x7fe2d80a4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f01073a0 0x7fe2f0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2f0104a70 0x7fe2f01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 --2- 172.21.15.99:0/1155298489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102140 0x7fe2f0101ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.120+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 >> 172.21.15.99:0/1155298489 conn(0x7fe2f00fbb30 msgr2=0x7fe2f0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.121+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 shutdown_connections 2024-03-23T23:40:33.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:33.121+0000 7fe2f8400640 1 -- 172.21.15.99:0/1155298489 wait complete. 2024-03-23T23:40:33.751 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 podman[69835]: 2024-03-23 23:40:33.469504239 +0000 UTC m=+0.174455083 container create 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-23T23:40:33.751 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 podman[69835]: 2024-03-23 23:40:33.750932092 +0000 UTC m=+0.455882942 container init 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:33 smithi099 ceph-mon[31715]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:33 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1155298489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 podman[69835]: 2024-03-23 23:40:33.761843543 +0000 UTC m=+0.466794394 container start 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 ceph-mds[69854]: -- 172.21.15.99:0/914055760 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb39ba9a40 con 0x55cb39b7cc00 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j[69850]: starting mds.j at 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 bash[69835]: 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba 2024-03-23T23:40:34.111 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:40:33 smithi099 systemd[1]: Started Ceph mds.j for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:33 smithi149 ceph-mon[36083]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:34.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:33 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1155298489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:33 smithi174 ceph-mon[37002]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:33 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1155298489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:35.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6834/3136510370,v1:172.21.15.99:6835/3136510370] up:boot 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: fsmap 3 up:standby 2024-03-23T23:40:35.242 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:34 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 -- 172.21.15.99:0/722299853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 msgr2=0x7f4a88100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 --2- 172.21.15.99:0/722299853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88100b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009900 tx=0x7f4a7402bc30 comp rx=0 tx=0).stop 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 -- 172.21.15.99:0/722299853 shutdown_connections 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 --2- 172.21.15.99:0/722299853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a88101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 --2- 172.21.15.99:0/722299853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88100b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 --2- 172.21.15.99:0/722299853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a881001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 -- 172.21.15.99:0/722299853 >> 172.21.15.99:0/722299853 conn(0x7f4a880fb980 msgr2=0x7f4a880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:35.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 -- 172.21.15.99:0/722299853 shutdown_connections 2024-03-23T23:40:35.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.369+0000 7f4a8d400640 1 -- 172.21.15.99:0/722299853 wait complete. 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 Processor -- start 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 -- start start 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a881a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810f350 con 0x7f4a88100720 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810f1d0 con 0x7f4a881010c0 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8d400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8810f4d0 con 0x7f4a880ffdc0 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a87400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50976/0 (socket says 172.21.15.99:50976) 2024-03-23T23:40:35.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a87400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58582/0 (socket says 172.21.15.99:58582) 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a87e00640 1 -- 172.21.15.99:0/2450153576 learned_addr learned my addr 172.21.15.99:0/2450153576 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.370+0000 7f4a8ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a881a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 -- 172.21.15.99:0/2450153576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 msgr2=0x7f4a88199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 -- 172.21.15.99:0/2450153576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 msgr2=0x7f4a881a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a881a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 -- 172.21.15.99:0/2450153576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a881a1620 con 0x7f4a88100720 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87e00640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a8ca00640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a881a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a87400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009230 tx=0x7f4a7402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041070 con 0x7f4a88100720 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a74004450 con 0x7f4a88100720 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a881a18b0 con 0x7f4a88100720 2024-03-23T23:40:35.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.371+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74003d80 con 0x7f4a88100720 2024-03-23T23:40:35.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.372+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a881a1da0 con 0x7f4a88100720 2024-03-23T23:40:35.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.373+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8810ab10 con 0x7f4a88100720 2024-03-23T23:40:35.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.375+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4a74044070 con 0x7f4a88100720 2024-03-23T23:40:35.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.376+0000 7f4a84c00640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 0x7f4a6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:35.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.376+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a740effe0 con 0x7f4a88100720 2024-03-23T23:40:35.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.376+0000 7f4a87e00640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 0x7f4a6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:35.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.377+0000 7f4a87e00640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 0x7f4a6c0a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011500 tx=0x7f4a78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:35.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.379+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a740bd4e0 con 0x7f4a88100720 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6834/3136510370,v1:172.21.15.99:6835/3136510370] up:boot 2024-03-23T23:40:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: fsmap 3 up:standby 2024-03-23T23:40:35.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:34 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6834/3136510370,v1:172.21.15.99:6835/3136510370] up:boot 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: fsmap 3 up:standby 2024-03-23T23:40:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:34 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T23:40:35.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.624+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8802d190 con 0x7f4a88100720 2024-03-23T23:40:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.625+0000 7f4a84c00640 1 -- 172.21.15.99:0/2450153576 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f4a740bcde0 con 0x7f4a88100720 2024-03-23T23:40:35.625 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:35.625 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:35.625 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 4 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 msgr2=0x7f4a6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 0x7f4a6c0a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a78011500 tx=0x7f4a78005f90 comp rx=0 tx=0).stop 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 msgr2=0x7f4a88199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009230 tx=0x7f4a7402c650 comp rx=0 tx=0).stop 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 shutdown_connections 2024-03-23T23:40:35.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4a6c0a2800 0x7f4a6c0a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881010c0 0x7f4a881a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a88100720 0x7f4a88199f40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 --2- 172.21.15.99:0/2450153576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a880ffdc0 0x7f4a88199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 >> 172.21.15.99:0/2450153576 conn(0x7f4a880fb980 msgr2=0x7f4a880fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 shutdown_connections 2024-03-23T23:40:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:35.628+0000 7f4a8d400640 1 -- 172.21.15.99:0/2450153576 wait complete. 2024-03-23T23:40:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:35 smithi099 ceph-mon[31715]: Deploying daemon mds.d on smithi099 2024-03-23T23:40:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:35 smithi099 ceph-mon[31715]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:35 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2450153576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:35 smithi149 ceph-mon[36083]: Deploying daemon mds.d on smithi099 2024-03-23T23:40:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:35 smithi149 ceph-mon[36083]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:35 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2450153576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:36.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:35 smithi174 ceph-mon[37002]: Deploying daemon mds.d on smithi099 2024-03-23T23:40:36.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:35 smithi174 ceph-mon[37002]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:36.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:35 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2450153576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:37.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:37.724 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 podman[70178]: 2024-03-23 23:40:37.500412565 +0000 UTC m=+0.157700733 container create 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T23:40:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 -- 172.21.15.99:0/906791026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 msgr2=0x7f3f7c0744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 --2- 172.21.15.99:0/906791026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c0744e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009860 tx=0x7f3f6802bc20 comp rx=0 tx=0).stop 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 -- 172.21.15.99:0/906791026 shutdown_connections 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 --2- 172.21.15.99:0/906791026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 0x7f3f7c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 --2- 172.21.15.99:0/906791026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c0744e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 --2- 172.21.15.99:0/906791026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 -- 172.21.15.99:0/906791026 >> 172.21.15.99:0/906791026 conn(0x7f3f7c0fcc50 msgr2=0x7f3f7c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 -- 172.21.15.99:0/906791026 shutdown_connections 2024-03-23T23:40:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.857+0000 7f3f82600640 1 -- 172.21.15.99:0/906791026 wait complete. 2024-03-23T23:40:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.858+0000 7f3f82600640 1 Processor -- start 2024-03-23T23:40:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.858+0000 7f3f82600640 1 -- start start 2024-03-23T23:40:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 0x7f3f7c1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c10f520 con 0x7f3f7c075660 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c10f3a0 con 0x7f3f7c109570 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f82600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c10f6a0 con 0x7f3f7c074080 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58626/0 (socket says 172.21.15.99:58626) 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 -- 172.21.15.99:0/2260519046 learned_addr learned my addr 172.21.15.99:0/2260519046 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7be00640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 0x7f3f7c1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7b400640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 -- 172.21.15.99:0/2260519046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 msgr2=0x7f3f7c199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 -- 172.21.15.99:0/2260519046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 msgr2=0x7f3f7c1a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 0x7f3f7c1a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:37.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.859+0000 7f3f7aa00640 1 -- 172.21.15.99:0/2260519046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c1a17b0 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f7aa00640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034a80 tx=0x7f3f680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68041030 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c1a1a40 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c1a1f80 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f6802ce50 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.860+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68039040 con 0x7f3f7c075660 2024-03-23T23:40:37.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.861+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0052c0 con 0x7f3f7c075660 2024-03-23T23:40:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.864+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3f68044070 con 0x7f3f7c075660 2024-03-23T23:40:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.866+0000 7f3f73e00640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 0x7f3f600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.866+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f680ef600 con 0x7f3f7c075660 2024-03-23T23:40:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.867+0000 7f3f7b400640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 0x7f3f600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.867+0000 7f3f7b400640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 0x7f3f600a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c005fd0 tx=0x7f3f6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:37.869+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f6804b070 con 0x7f3f7c075660 2024-03-23T23:40:38.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:38 smithi099 ceph-mon[31715]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:38 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 podman[70178]: 2024-03-23 23:40:37.78838955 +0000 UTC m=+0.445677736 container init 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 podman[70178]: 2024-03-23 23:40:37.795386368 +0000 UTC m=+0.452674499 container start 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 ceph-mds[70217]: -- 172.21.15.99:0/1522954770 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b7ca35a40 con 0x559b7ca08800 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d[70193]: starting mds.d at 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 bash[70178]: 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b 2024-03-23T23:40:38.010 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:40:37 smithi099 systemd[1]: Started Ceph mds.d for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:38.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.114+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f5c0055b0 con 0x7f3f7c075660 2024-03-23T23:40:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.116+0000 7f3f73e00640 1 -- 172.21.15.99:0/2260519046 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f3f680bcb20 con 0x7f3f7c075660 2024-03-23T23:40:38.116 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:38.116 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:38.116 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 5 2024-03-23T23:40:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 msgr2=0x7f3f600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 0x7f3f600a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c005fd0 tx=0x7f3f6c009260 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 msgr2=0x7f3f7c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f68034a80 tx=0x7f3f680042f0 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 shutdown_connections 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3f600a2880 0x7f3f600a4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f7c109570 0x7f3f7c1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f7c075660 0x7f3f7c19a0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 --2- 172.21.15.99:0/2260519046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f7c074080 0x7f3f7c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.118+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 >> 172.21.15.99:0/2260519046 conn(0x7f3f7c0fcc50 msgr2=0x7f3f7c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.119+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 shutdown_connections 2024-03-23T23:40:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:38.119+0000 7f3f82600640 1 -- 172.21.15.99:0/2260519046 wait complete. 2024-03-23T23:40:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:38 smithi149 ceph-mon[36083]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:38 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:38.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:38 smithi174 ceph-mon[37002]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:38.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:38 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6836/2449593351,v1:172.21.15.99:6837/2449593351] up:boot 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: fsmap 4 up:standby 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:39.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:39 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2260519046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6836/2449593351,v1:172.21.15.99:6837/2449593351] up:boot 2024-03-23T23:40:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: fsmap 4 up:standby 2024-03-23T23:40:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T23:40:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:39.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:39.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:39.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:39 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2260519046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6836/2449593351,v1:172.21.15.99:6837/2449593351] up:boot 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: fsmap 4 up:standby 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:39.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:39.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:39 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2260519046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:40.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:40.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:40 smithi149 ceph-mon[36083]: Deploying daemon mds.e on smithi149 2024-03-23T23:40:40.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:40 smithi149 ceph-mon[36083]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:40 smithi099 ceph-mon[31715]: Deploying daemon mds.e on smithi149 2024-03-23T23:40:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:40 smithi099 ceph-mon[31715]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 -- 172.21.15.99:0/3399270479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980772a0 msgr2=0x7f4d98077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/3399270479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980772a0 0x7f4d98077700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009860 tx=0x7f4d8002bb70 comp rx=0 tx=0).stop 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 -- 172.21.15.99:0/3399270479 shutdown_connections 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/3399270479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98077c40 0x7f4d980784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/3399270479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d980772a0 0x7f4d98077700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/3399270479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d98076060 0x7f4d98076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 -- 172.21.15.99:0/3399270479 >> 172.21.15.99:0/3399270479 conn(0x7f4d980fee50 msgr2=0x7f4d98101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.361+0000 7f4d9ca00640 1 -- 172.21.15.99:0/3399270479 shutdown_connections 2024-03-23T23:40:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.362+0000 7f4d9ca00640 1 -- 172.21.15.99:0/3399270479 wait complete. 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.362+0000 7f4d9ca00640 1 Processor -- start 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 -- start start 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d980772a0 0x7f4d9819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 0x7f4d981a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98113970 con 0x7f4d98077c40 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981137f0 con 0x7f4d98076060 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d9ca00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d98113af0 con 0x7f4d980772a0 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d95e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d95e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:33300/0 (socket says 172.21.15.99:33300) 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d95e00640 1 -- 172.21.15.99:0/528375390 learned_addr learned my addr 172.21.15.99:0/528375390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d96800640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 0x7f4d981a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.363+0000 7f4d8fe00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d980772a0 0x7f4d9819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 -- 172.21.15.99:0/528375390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d980772a0 msgr2=0x7f4d9819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d980772a0 0x7f4d9819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 -- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 msgr2=0x7f4d981a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 0x7f4d981a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 -- 172.21.15.99:0/528375390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d981a5c40 con 0x7f4d98076060 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d96800640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 0x7f4d981a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d95e00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800cd80 con 0x7f4d98076060 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d88005500 con 0x7f4d98076060 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d981a5f30 con 0x7f4d98076060 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88010690 con 0x7f4d98076060 2024-03-23T23:40:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.364+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d981a6340 con 0x7f4d98076060 2024-03-23T23:40:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.366+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98110450 con 0x7f4d98076060 2024-03-23T23:40:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.367+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4d880046c0 con 0x7f4d98076060 2024-03-23T23:40:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.368+0000 7f4d8ea00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.369+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d880ceb10 con 0x7f4d98076060 2024-03-23T23:40:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.369+0000 7f4d8fe00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.369+0000 7f4d8fe00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 0x7f4d780a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009830 tx=0x7f4d80034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:40.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.373+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d8809c050 con 0x7f4d98076060 2024-03-23T23:40:40.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:40 smithi174 ceph-mon[37002]: Deploying daemon mds.e on smithi149 2024-03-23T23:40:40.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:40 smithi174 ceph-mon[37002]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:40.590 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 systemd[1]: Starting Ceph mds.e for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.612+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d9802d070 con 0x7f4d98076060 2024-03-23T23:40:40.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.613+0000 7f4d8ea00640 1 -- 172.21.15.99:0/528375390 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f4d8809b950 con 0x7f4d98076060 2024-03-23T23:40:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:40.614 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-23T23:40:40.614 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 5 2024-03-23T23:40:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 msgr2=0x7f4d780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 0x7f4d780a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009830 tx=0x7f4d80034ad0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 msgr2=0x7f4d9819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800edf0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 shutdown_connections 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d780a2800 0x7f4d780a4cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d98077c40 0x7f4d981a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d980772a0 0x7f4d9819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 --2- 172.21.15.99:0/528375390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d98076060 0x7f4d9819e020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.616+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 >> 172.21.15.99:0/528375390 conn(0x7f4d980fee50 msgr2=0x7f4d981005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.617+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 shutdown_connections 2024-03-23T23:40:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:40.617+0000 7f4d9ca00640 1 -- 172.21.15.99:0/528375390 wait complete. 2024-03-23T23:40:40.843 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 podman[65554]: 2024-03-23 23:40:40.494065739 +0000 UTC m=+0.017999144 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:40.844 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 podman[65554]: 2024-03-23 23:40:40.605479521 +0000 UTC m=+0.129412906 container create ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-23T23:40:40.844 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 podman[65554]: 2024-03-23 23:40:40.841196076 +0000 UTC m=+0.365129468 container init ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:40:40.844 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 podman[65554]: 2024-03-23 23:40:40.844085819 +0000 UTC m=+0.368019214 container start ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5) 2024-03-23T23:40:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:41 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/528375390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:41.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:41.144 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 ceph-mds[65573]: -- 172.21.15.149:0/2935988065 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5615fe6c1a40 con 0x5615fe694800 2024-03-23T23:40:41.145 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: starting mds.e at 2024-03-23T23:40:41.145 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 bash[65554]: ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e 2024-03-23T23:40:41.145 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:40:40 smithi149 systemd[1]: Started Ceph mds.e for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:41.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:41 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/528375390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:41.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:41.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:41.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:41 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/528375390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:41.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:41.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6834/1177424904,v1:172.21.15.149:6835/1177424904] up:boot 2024-03-23T23:40:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: fsmap 5 up:standby 2024-03-23T23:40:42.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:40:42.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:42.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:42.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:42.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6834/1177424904,v1:172.21.15.149:6835/1177424904] up:boot 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: fsmap 5 up:standby 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:42.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6834/1177424904,v1:172.21.15.149:6835/1177424904] up:boot 2024-03-23T23:40:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: fsmap 5 up:standby 2024-03-23T23:40:42.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:40:42.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:42.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:42.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:42.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:42.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.858+0000 7f8274e00640 1 -- 172.21.15.99:0/3818870829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8270103270 msgr2=0x7f8270109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.858+0000 7f8274e00640 1 --2- 172.21.15.99:0/3818870829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8270103270 0x7f8270109b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8268009ca0 tx=0x7f826802c100 comp rx=0 tx=0).stop 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 -- 172.21.15.99:0/3818870829 shutdown_connections 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 --2- 172.21.15.99:0/3818870829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8270103270 0x7f8270109b00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 --2- 172.21.15.99:0/3818870829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701028b0 0x7f8270102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 --2- 172.21.15.99:0/3818870829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82701016c0 0x7f8270101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:42.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 -- 172.21.15.99:0/3818870829 >> 172.21.15.99:0/3818870829 conn(0x7f82700fcc50 msgr2=0x7f82700ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:42.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 -- 172.21.15.99:0/3818870829 shutdown_connections 2024-03-23T23:40:42.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.859+0000 7f8274e00640 1 -- 172.21.15.99:0/3818870829 wait complete. 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 Processor -- start 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 -- start start 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 0x7f827019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 0x7f82701a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010d340 con 0x7f8270103270 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010d1c0 con 0x7f82701028b0 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f8274e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827010d4c0 con 0x7f82701016c0 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f826de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f826de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52804/0 (socket says 172.21.15.99:52804) 2024-03-23T23:40:42.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.860+0000 7f826de00640 1 -- 172.21.15.99:0/2411443883 learned_addr learned my addr 172.21.15.99:0/2411443883 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826d400640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 0x7f827019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826e800640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 0x7f82701a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 -- 172.21.15.99:0/2411443883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 msgr2=0x7f827019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 0x7f827019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 -- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 msgr2=0x7f82701a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 0x7f82701a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 -- 172.21.15.99:0/2411443883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82701a17a0 con 0x7f82701016c0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826e800640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 0x7f82701a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826d400640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 0x7f827019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f826de00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9c0 tx=0x7f826000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.861+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000cde0 con 0x7f82701016c0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.862+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82701a1a90 con 0x7f82701016c0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.862+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8260010910 con 0x7f82701016c0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.862+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260018a10 con 0x7f82701016c0 2024-03-23T23:40:42.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.862+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701a1fe0 con 0x7f82701016c0 2024-03-23T23:40:42.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.863+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82701a2210 con 0x7f82701016c0 2024-03-23T23:40:42.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.865+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f826001f020 con 0x7f82701016c0 2024-03-23T23:40:42.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.866+0000 7f8266a00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 0x7f82540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:42.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.867+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82600cef90 con 0x7f82701016c0 2024-03-23T23:40:42.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.867+0000 7f826d400640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 0x7f82540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:42.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.867+0000 7f826d400640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 0x7f82540a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f827019b090 tx=0x7f825c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:42.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:42.869+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f826009c430 con 0x7f82701016c0 2024-03-23T23:40:43.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.110+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82701a23e0 con 0x7f82701016c0 2024-03-23T23:40:43.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.111+0000 7f8266a00640 1 -- 172.21.15.99:0/2411443883 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f8260020e40 con 0x7f82701016c0 2024-03-23T23:40:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:43.112 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-23T23:40:43.112 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-03-23T23:40:43.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 msgr2=0x7f82540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:43.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 0x7f82540a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f827019b090 tx=0x7f825c005f90 comp rx=0 tx=0).stop 2024-03-23T23:40:43.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 msgr2=0x7f8270199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:43.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f826000e9c0 tx=0x7f826000ee80 comp rx=0 tx=0).stop 2024-03-23T23:40:43.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 shutdown_connections 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82540a2800 0x7f82540a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8270103270 0x7f82701a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82701028b0 0x7f827019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 --2- 172.21.15.99:0/2411443883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82701016c0 0x7f8270199b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 >> 172.21.15.99:0/2411443883 conn(0x7f82700fcc50 msgr2=0x7f82700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 shutdown_connections 2024-03-23T23:40:43.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:43.114+0000 7f8274e00640 1 -- 172.21.15.99:0/2411443883 wait complete. 2024-03-23T23:40:43.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:43 smithi099 ceph-mon[31715]: Deploying daemon mds.f on smithi174 2024-03-23T23:40:43.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:43 smithi149 ceph-mon[36083]: Deploying daemon mds.f on smithi174 2024-03-23T23:40:43.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:43 smithi174 ceph-mon[37002]: Deploying daemon mds.f on smithi174 2024-03-23T23:40:43.423 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 systemd[1]: Starting Ceph mds.f for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:43.716 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 podman[67492]: 2024-03-23 23:40:43.444895054 +0000 UTC m=+0.092781415 container create 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T23:40:43.717 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 podman[67492]: 2024-03-23 23:40:43.370162612 +0000 UTC m=+0.018048924 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:43.717 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 podman[67492]: 2024-03-23 23:40:43.717334052 +0000 UTC m=+0.365220351 container init 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:40:44.046 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 podman[67492]: 2024-03-23 23:40:43.719959162 +0000 UTC m=+0.367845455 container start 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-23T23:40:44.047 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 ceph-mds[67510]: -- 172.21.15.174:0/4059425465 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc8206ba40 con 0x55cc8203f000 2024-03-23T23:40:44.047 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: starting mds.f at 2024-03-23T23:40:44.047 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 bash[67492]: 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 2024-03-23T23:40:44.047 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:40:43 smithi174 systemd[1]: Started Ceph mds.f for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2411443883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:44 smithi099 ceph-mon[31715]: Deploying daemon mds.c on smithi174 2024-03-23T23:40:44.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:44.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2411443883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:44.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:44 smithi149 ceph-mon[36083]: Deploying daemon mds.c on smithi174 2024-03-23T23:40:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2411443883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:44.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:44 smithi174 ceph-mon[37002]: Deploying daemon mds.c on smithi174 2024-03-23T23:40:45.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:45.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:45 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6834/731827456,v1:172.21.15.174:6835/731827456] up:boot 2024-03-23T23:40:45.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:45 smithi174 ceph-mon[37002]: fsmap 6 up:standby 2024-03-23T23:40:45.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:45 smithi174 ceph-mon[37002]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 -- 172.21.15.99:0/2445362969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c076060 msgr2=0x7f264c076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 --2- 172.21.15.99:0/2445362969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c076060 0x7f264c076480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f263c009900 tx=0x7f263c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 -- 172.21.15.99:0/2445362969 shutdown_connections 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 --2- 172.21.15.99:0/2445362969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c077c40 0x7f264c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 --2- 172.21.15.99:0/2445362969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c0772a0 0x7f264c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 --2- 172.21.15.99:0/2445362969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c076060 0x7f264c076480 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 -- 172.21.15.99:0/2445362969 >> 172.21.15.99:0/2445362969 conn(0x7f264c0fee50 msgr2=0x7f264c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 -- 172.21.15.99:0/2445362969 shutdown_connections 2024-03-23T23:40:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.357+0000 7f2652000640 1 -- 172.21.15.99:0/2445362969 wait complete. 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 Processor -- start 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 -- start start 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c076060 0x7f264c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 0x7f264c1a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1139b0 con 0x7f264c0772a0 2024-03-23T23:40:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c113830 con 0x7f264c076060 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2652000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c113b30 con 0x7f264c077c40 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2643e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2643e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50356/0 (socket says 172.21.15.99:50356) 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f2643e00640 1 -- 172.21.15.99:0/1114532918 learned_addr learned my addr 172.21.15.99:0/1114532918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.359+0000 7f264b400640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c076060 0x7f264c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f264be00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 0x7f264c1a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 -- 172.21.15.99:0/1114532918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 msgr2=0x7f264c1a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 0x7f264c1a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 -- 172.21.15.99:0/1114532918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c076060 msgr2=0x7f264c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c076060 0x7f264c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 -- 172.21.15.99:0/1114532918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c1a5c00 con 0x7f264c0772a0 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f264be00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 0x7f264c1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2643e00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9f0 tx=0x7f263400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400cd80 con 0x7f264c0772a0 2024-03-23T23:40:45.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2634010910 con 0x7f264c0772a0 2024-03-23T23:40:45.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.360+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c1a5ef0 con 0x7f264c0772a0 2024-03-23T23:40:45.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.361+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634018990 con 0x7f264c0772a0 2024-03-23T23:40:45.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.361+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1a63b0 con 0x7f264c0772a0 2024-03-23T23:40:45.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:45 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6834/731827456,v1:172.21.15.174:6835/731827456] up:boot 2024-03-23T23:40:45.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:45 smithi099 ceph-mon[31715]: fsmap 6 up:standby 2024-03-23T23:40:45.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:45 smithi099 ceph-mon[31715]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:45.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.362+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c1103f0 con 0x7f264c0772a0 2024-03-23T23:40:45.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.364+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2634010ac0 con 0x7f264c0772a0 2024-03-23T23:40:45.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.366+0000 7f2648c00640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 0x7f262c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:45.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.366+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26340cee40 con 0x7f264c0772a0 2024-03-23T23:40:45.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.366+0000 7f264b400640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 0x7f262c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:45.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.367+0000 7f264b400640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 0x7f262c0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f263c02c240 tx=0x7f263c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:45.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.369+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f263409c2e0 con 0x7f264c0772a0 2024-03-23T23:40:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:45 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6834/731827456,v1:172.21.15.174:6835/731827456] up:boot 2024-03-23T23:40:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:45 smithi149 ceph-mon[36083]: fsmap 6 up:standby 2024-03-23T23:40:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:40:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:45 smithi149 ceph-mon[36083]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:45.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.606+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f264c02d070 con 0x7f264c0772a0 2024-03-23T23:40:45.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.607+0000 7f2648c00640 1 -- 172.21.15.99:0/1114532918 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f263409bbe0 con 0x7f264c0772a0 2024-03-23T23:40:45.607 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:45.607 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-23T23:40:45.607 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 7 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 msgr2=0x7f262c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 0x7f262c0a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f263c02c240 tx=0x7f263c02df90 comp rx=0 tx=0).stop 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 msgr2=0x7f264c19e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9f0 tx=0x7f263400eeb0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 shutdown_connections 2024-03-23T23:40:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f262c0a2800 0x7f262c0a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f264c077c40 0x7f264c1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264c0772a0 0x7f264c19e520 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 --2- 172.21.15.99:0/1114532918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264c076060 0x7f264c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 >> 172.21.15.99:0/1114532918 conn(0x7f264c0fee50 msgr2=0x7f264c100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 shutdown_connections 2024-03-23T23:40:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:45.610+0000 7f2652000640 1 -- 172.21.15.99:0/1114532918 wait complete. 2024-03-23T23:40:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:46 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1114532918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:46.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:46 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1114532918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:46.375 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 systemd[1]: Starting Ceph mds.c for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:46.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:46 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1114532918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:46.628 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 podman[67796]: 2024-03-23 23:40:46.375550434 +0000 UTC m=+0.113973932 container create 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:40:46.628 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 podman[67796]: 2024-03-23 23:40:46.279111334 +0000 UTC m=+0.017534810 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:46.628 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 podman[67796]: 2024-03-23 23:40:46.628398512 +0000 UTC m=+0.366821981 container init 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-23T23:40:46.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 podman[67796]: 2024-03-23 23:40:46.631204591 +0000 UTC m=+0.369628060 container start 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-23T23:40:46.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 ceph-mds[67814]: -- 172.21.15.174:0/1367908991 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fdfb1d7a40 con 0x55fdfb1aa800 2024-03-23T23:40:46.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: starting mds.c at 2024-03-23T23:40:46.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 bash[67796]: 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 2024-03-23T23:40:46.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:40:46 smithi174 systemd[1]: Started Ceph mds.c for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:47.629 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.846+0000 7f63d6600640 1 -- 172.21.15.99:0/3625703275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 msgr2=0x7f63d0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.846+0000 7f63d6600640 1 --2- 172.21.15.99:0/3625703275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0103d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009900 tx=0x7f63c002bc30 comp rx=0 tx=0).stop 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 -- 172.21.15.99:0/3625703275 shutdown_connections 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 --2- 172.21.15.99:0/3625703275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0105490 0x7f63d010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 --2- 172.21.15.99:0/3625703275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0104ad0 0x7f63d0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 --2- 172.21.15.99:0/3625703275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0103d00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 -- 172.21.15.99:0/3625703275 >> 172.21.15.99:0/3625703275 conn(0x7f63d00fee50 msgr2=0x7f63d0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 -- 172.21.15.99:0/3625703275 shutdown_connections 2024-03-23T23:40:47.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.847+0000 7f63d6600640 1 -- 172.21.15.99:0/3625703275 wait complete. 2024-03-23T23:40:47.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 Processor -- start 2024-03-23T23:40:47.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 -- start start 2024-03-23T23:40:47.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:47.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0104ad0 0x7f63d019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:47.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0105490 0x7f63d01a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d010f5a0 con 0x7f63d0104ad0 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d010f420 con 0x7f63d0105490 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63d6600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d010f720 con 0x7f63d01038e0 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.848+0000 7f63cf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52838/0 (socket says 172.21.15.99:52838) 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 -- 172.21.15.99:0/4248374673 learned_addr learned my addr 172.21.15.99:0/4248374673 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63c7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0104ad0 0x7f63d019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cfe00640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0105490 0x7f63d01a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 -- 172.21.15.99:0/4248374673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0105490 msgr2=0x7f63d01a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0105490 0x7f63d01a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 -- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0104ad0 msgr2=0x7f63d019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:47.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0104ad0 0x7f63d019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 -- 172.21.15.99:0/4248374673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d01a1870 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63cf400640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63c0000c00 tx=0x7f63c002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0043020 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63c00054e0 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0003c40 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.849+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d01a1b00 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.850+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d01a2000 con 0x7f63d01038e0 2024-03-23T23:40:47.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.851+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d0068490 con 0x7f63d01038e0 2024-03-23T23:40:47.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.852+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f63c0046020 con 0x7f63d01038e0 2024-03-23T23:40:47.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.854+0000 7f63ccc00640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 0x7f63b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:47.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.854+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63c00f0b70 con 0x7f63d01038e0 2024-03-23T23:40:47.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.854+0000 7f63c7e00640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 0x7f63b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:47.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.855+0000 7f63c7e00640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 0x7f63b40a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009bd0 tx=0x7f63b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:47.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:47.857+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63c004d020 con 0x7f63d01038e0 2024-03-23T23:40:48.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.095+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63d002d110 con 0x7f63d01038e0 2024-03-23T23:40:48.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.096+0000 7f63ccc00640 1 -- 172.21.15.99:0/4248374673 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f63c00be010 con 0x7f63d01038e0 2024-03-23T23:40:48.096 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:48.096 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-23T23:40:48.097 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 msgr2=0x7f63b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 0x7f63b40a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009bd0 tx=0x7f63b8005f90 comp rx=0 tx=0).stop 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 msgr2=0x7f63d0199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63c0000c00 tx=0x7f63c002c650 comp rx=0 tx=0).stop 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 shutdown_connections 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f63b40a2880 0x7f63b40a4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:48.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63d0105490 0x7f63d01a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:48.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d0104ad0 0x7f63d019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:48.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 --2- 172.21.15.99:0/4248374673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d01038e0 0x7f63d0199c60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:48.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 >> 172.21.15.99:0/4248374673 conn(0x7f63d00fee50 msgr2=0x7f63d0107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:48.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 shutdown_connections 2024-03-23T23:40:48.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:48.099+0000 7f63d6600640 1 -- 172.21.15.99:0/4248374673 wait complete. 2024-03-23T23:40:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:48.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: Deploying daemon mds.h on smithi149 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:boot 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: fsmap 7 up:standby 2024-03-23T23:40:48.115 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: Deploying daemon mds.h on smithi149 2024-03-23T23:40:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:boot 2024-03-23T23:40:48.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: fsmap 7 up:standby 2024-03-23T23:40:48.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: Deploying daemon mds.h on smithi149 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:boot 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: fsmap 7 up:standby 2024-03-23T23:40:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:40:49.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:48 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4248374673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:49.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:48 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4248374673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:49.144 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:48 smithi149 systemd[1]: Starting Ceph mds.h for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:48 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4248374673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:49.462 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 podman[65864]: 2024-03-23 23:40:49.092591574 +0000 UTC m=+0.017917014 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:49.463 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 podman[65864]: 2024-03-23 23:40:49.202492899 +0000 UTC m=+0.127818332 container create d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:40:49.463 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 podman[65864]: 2024-03-23 23:40:49.463495434 +0000 UTC m=+0.388820865 container init d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-23T23:40:49.806 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 podman[65864]: 2024-03-23 23:40:49.466290627 +0000 UTC m=+0.391616054 container start d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-23T23:40:49.807 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mds[65883]: -- 172.21.15.149:0/3917713515 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b82a7bfa40 con 0x55b82a793000 2024-03-23T23:40:49.807 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: starting mds.h at 2024-03-23T23:40:49.807 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 bash[65864]: d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b 2024-03-23T23:40:49.807 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:40:49 smithi149 systemd[1]: Started Ceph mds.h for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:50.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:50.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:50.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:50.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:50.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:50.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 -- 172.21.15.99:0/577009420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 msgr2=0x7f71701016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 --2- 172.21.15.99:0/577009420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701016d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7168009ca0 tx=0x7f716802c0f0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 -- 172.21.15.99:0/577009420 shutdown_connections 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 --2- 172.21.15.99:0/577009420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701016d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 --2- 172.21.15.99:0/577009420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71701008b0 0x7f7170100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 --2- 172.21.15.99:0/577009420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71700fff50 0x7f7170100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.338+0000 7f7178000640 1 -- 172.21.15.99:0/577009420 >> 172.21.15.99:0/577009420 conn(0x7f71700fbb50 msgr2=0x7f71700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:50.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.339+0000 7f7178000640 1 -- 172.21.15.99:0/577009420 shutdown_connections 2024-03-23T23:40:50.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.339+0000 7f7178000640 1 -- 172.21.15.99:0/577009420 wait complete. 2024-03-23T23:40:50.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.339+0000 7f7178000640 1 Processor -- start 2024-03-23T23:40:50.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 -- start start 2024-03-23T23:40:50.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:50.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71701008b0 0x7f717019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717010f550 con 0x7f71701008b0 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717010f3d0 con 0x7f7170101250 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7178000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717010f6d0 con 0x7f71700fff50 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7175600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7174c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71701008b0 0x7f717019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7175600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40718/0 (socket says 172.21.15.99:40718) 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7175600640 1 -- 172.21.15.99:0/4194172857 learned_addr learned my addr 172.21.15.99:0/4194172857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.340+0000 7f7176000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 -- 172.21.15.99:0/4194172857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 msgr2=0x7f71701a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 -- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71701008b0 msgr2=0x7f717019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:50.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71701008b0 0x7f717019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 -- 172.21.15.99:0/4194172857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71701a17b0 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7175600640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f716400d950 tx=0x7f716400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164017030 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71701a1aa0 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7164004020 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640047a0 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.341+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71701a1ff0 con 0x7f71700fff50 2024-03-23T23:40:50.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.342+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717010ae30 con 0x7f71700fff50 2024-03-23T23:40:50.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.344+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f71640041d0 con 0x7f71700fff50 2024-03-23T23:40:50.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.345+0000 7f716e000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 0x7f71580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:50.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.346+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71640ce000 con 0x7f71700fff50 2024-03-23T23:40:50.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.346+0000 7f7174c00640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 0x7f71580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:50.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.346+0000 7f7174c00640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 0x7f71580a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f717019b0a0 tx=0x7f715c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:50.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.350+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f716409b4a0 con 0x7f71700fff50 2024-03-23T23:40:50.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.590+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f717002d070 con 0x7f71700fff50 2024-03-23T23:40:50.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.591+0000 7f716e000640 1 -- 172.21.15.99:0/4194172857 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f716409ada0 con 0x7f71700fff50 2024-03-23T23:40:50.591 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:50.591 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-23T23:40:50.591 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 9 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.594+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 msgr2=0x7f71580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.594+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 0x7f71580a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f717019b0a0 tx=0x7f715c005f90 comp rx=0 tx=0).stop 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.594+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 msgr2=0x7f7170199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.594+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f716400d950 tx=0x7f716400de10 comp rx=0 tx=0).stop 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 shutdown_connections 2024-03-23T23:40:50.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f71580a2800 0x7f71580a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7170101250 0x7f71701a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71701008b0 0x7f717019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 --2- 172.21.15.99:0/4194172857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71700fff50 0x7f7170199b90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 >> 172.21.15.99:0/4194172857 conn(0x7f71700fbb50 msgr2=0x7f71700fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 shutdown_connections 2024-03-23T23:40:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:50.595+0000 7f7178000640 1 -- 172.21.15.99:0/4194172857 wait complete. 2024-03-23T23:40:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:50 smithi099 ceph-mon[31715]: Deploying daemon mds.i on smithi174 2024-03-23T23:40:50.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:50 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6836/3655333447,v1:172.21.15.149:6837/3655333447] up:boot 2024-03-23T23:40:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:50 smithi099 ceph-mon[31715]: fsmap 8 up:standby 2024-03-23T23:40:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:40:50.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:50 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4194172857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:50 smithi149 ceph-mon[36083]: Deploying daemon mds.i on smithi174 2024-03-23T23:40:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:50 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6836/3655333447,v1:172.21.15.149:6837/3655333447] up:boot 2024-03-23T23:40:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:50 smithi149 ceph-mon[36083]: fsmap 8 up:standby 2024-03-23T23:40:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:40:51.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:50 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4194172857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:51.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:50 smithi174 ceph-mon[37002]: Deploying daemon mds.i on smithi174 2024-03-23T23:40:51.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:50 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6836/3655333447,v1:172.21.15.149:6837/3655333447] up:boot 2024-03-23T23:40:51.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:50 smithi174 ceph-mon[37002]: fsmap 8 up:standby 2024-03-23T23:40:51.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:40:51.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:50 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4194172857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:51 smithi099 ceph-mon[31715]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:51 smithi149 ceph-mon[36083]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:52.170 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:51 smithi174 ceph-mon[37002]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:52.171 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:51 smithi174 systemd[1]: Starting Ceph mds.i for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:40:52.422 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 podman[68106]: 2024-03-23 23:40:52.073698919 +0000 UTC m=+0.017979060 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:40:52.423 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 podman[68106]: 2024-03-23 23:40:52.195676195 +0000 UTC m=+0.139956342 container create 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-23T23:40:52.423 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 podman[68106]: 2024-03-23 23:40:52.42330636 +0000 UTC m=+0.367586491 container init 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:40:52.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:52.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 podman[68106]: 2024-03-23 23:40:52.427784671 +0000 UTC m=+0.372064791 container start 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-23T23:40:52.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 ceph-mds[68127]: -- 172.21.15.174:0/4054338485 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561abea27a40 con 0x561abe9fa800 2024-03-23T23:40:52.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: starting mds.i at 2024-03-23T23:40:52.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 bash[68106]: 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 2024-03-23T23:40:52.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:40:52 smithi174 systemd[1]: Started Ceph mds.i for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:52.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 -- 172.21.15.99:0/1194847943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478104ad0 msgr2=0x7fb478104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:52.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 --2- 172.21.15.99:0/1194847943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478104ad0 0x7fb478104f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb464009860 tx=0x7fb46402bb70 comp rx=0 tx=0).stop 2024-03-23T23:40:52.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 -- 172.21.15.99:0/1194847943 shutdown_connections 2024-03-23T23:40:52.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 --2- 172.21.15.99:0/1194847943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478105490 0x7fb47810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:52.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 --2- 172.21.15.99:0/1194847943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478104ad0 0x7fb478104f50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:52.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 --2- 172.21.15.99:0/1194847943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:52.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 -- 172.21.15.99:0/1194847943 >> 172.21.15.99:0/1194847943 conn(0x7fb4780fee90 msgr2=0x7fb4781012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:52.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 -- 172.21.15.99:0/1194847943 shutdown_connections 2024-03-23T23:40:52.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.824+0000 7fb47d800640 1 -- 172.21.15.99:0/1194847943 wait complete. 2024-03-23T23:40:52.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.825+0000 7fb47d800640 1 Processor -- start 2024-03-23T23:40:52.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.825+0000 7fb47d800640 1 -- start start 2024-03-23T23:40:52.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478104ad0 0x7fb47819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 0x7fb4781a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47810f560 con 0x7fb4781038e0 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47810f3e0 con 0x7fb478104ad0 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb47d800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47810f6e0 con 0x7fb478105490 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb476a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb476a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56752/0 (socket says 172.21.15.99:56752) 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb476a00640 1 -- 172.21.15.99:0/1359205006 learned_addr learned my addr 172.21.15.99:0/1359205006 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb477400640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 0x7fb4781a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.826+0000 7fb476000640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478104ad0 0x7fb47819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 -- 172.21.15.99:0/1359205006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 msgr2=0x7fb4781a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:52.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 0x7fb4781a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 -- 172.21.15.99:0/1359205006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478104ad0 msgr2=0x7fb47819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478104ad0 0x7fb47819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 -- 172.21.15.99:0/1359205006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4781a17c0 con 0x7fb4781038e0 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb477400640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 0x7fb4781a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb476a00640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800ccc0 con 0x7fb4781038e0 2024-03-23T23:40:52.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4781a1ab0 con 0x7fb4781038e0 2024-03-23T23:40:52.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4680035f0 con 0x7fb4781038e0 2024-03-23T23:40:52.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800ce60 con 0x7fb4781038e0 2024-03-23T23:40:52.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.827+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4781a2020 con 0x7fb4781038e0 2024-03-23T23:40:52.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.829+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47810ac00 con 0x7fb4781038e0 2024-03-23T23:40:52.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.830+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb4680037a0 con 0x7fb4781038e0 2024-03-23T23:40:52.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.832+0000 7fb46f400640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 0x7fb45c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:52.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.832+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb4680ce8c0 con 0x7fb4781038e0 2024-03-23T23:40:52.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.832+0000 7fb476000640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 0x7fb45c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:52.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.833+0000 7fb476000640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 0x7fb45c0a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb464009830 tx=0x7fb464034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:52.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:52.841+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb46809bd60 con 0x7fb4781038e0 2024-03-23T23:40:53.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.079+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb47802d070 con 0x7fb4781038e0 2024-03-23T23:40:53.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.081+0000 7fb46f400640 1 -- 172.21.15.99:0/1359205006 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7fb47802d070 con 0x7fb4781038e0 2024-03-23T23:40:53.080 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:53.080 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-23T23:40:53.081 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 10 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 msgr2=0x7fb45c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 0x7fb45c0a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb464009830 tx=0x7fb464034ab0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 msgr2=0x7fb478199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 shutdown_connections 2024-03-23T23:40:53.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb45c0a2800 0x7fb45c0a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb478105490 0x7fb4781a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb478104ad0 0x7fb47819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 --2- 172.21.15.99:0/1359205006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4781038e0 0x7fb478199ba0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 >> 172.21.15.99:0/1359205006 conn(0x7fb4780fee90 msgr2=0x7fb478107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 shutdown_connections 2024-03-23T23:40:53.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:53.084+0000 7fb47d800640 1 -- 172.21.15.99:0/1359205006 wait complete. 2024-03-23T23:40:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:53.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:53.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: Deploying daemon mds.g on smithi099 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:boot 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: fsmap 9 up:standby 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:40:53.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1359205006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: Deploying daemon mds.g on smithi099 2024-03-23T23:40:53.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:53.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:boot 2024-03-23T23:40:53.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: fsmap 9 up:standby 2024-03-23T23:40:53.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:40:53.863 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1359205006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:53.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: Deploying daemon mds.g on smithi099 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:boot 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: fsmap 9 up:standby 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:40:53.895 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1359205006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:55.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:55.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 -- 172.21.15.99:0/3285332064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4096810 msgr2=0x7fa7f4098c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:55.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 --2- 172.21.15.99:0/3285332064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4096810 0x7fa7f4098c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009860 tx=0x7fa7ec02bb70 comp rx=0 tx=0).stop 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 -- 172.21.15.99:0/3285332064 shutdown_connections 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 --2- 172.21.15.99:0/3285332064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f409ba70 0x7fa7f409dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 --2- 172.21.15.99:0/3285332064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f4099140 0x7fa7f409b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 --2- 172.21.15.99:0/3285332064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4096810 0x7fa7f4098c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.567+0000 7fa800a00640 1 -- 172.21.15.99:0/3285332064 >> 172.21.15.99:0/3285332064 conn(0x7fa7f40901e0 msgr2=0x7fa7f4092640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.568+0000 7fa800a00640 1 -- 172.21.15.99:0/3285332064 shutdown_connections 2024-03-23T23:40:55.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.568+0000 7fa800a00640 1 -- 172.21.15.99:0/3285332064 wait complete. 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 Processor -- start 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 -- start start 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4099140 0x7fa7f412e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f409ba70 0x7fa7f4135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40a1a10 con 0x7fa7f409ba70 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40a1890 con 0x7fa7f4099140 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.569+0000 7fa800a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40a1b90 con 0x7fa7f4096810 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:55.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fb200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f409ba70 0x7fa7f4135810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40764/0 (socket says 172.21.15.99:40764) 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 -- 172.21.15.99:0/339128497 learned_addr learned my addr 172.21.15.99:0/339128497 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7f9e00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4099140 0x7fa7f412e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 -- 172.21.15.99:0/339128497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4099140 msgr2=0x7fa7f412e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4099140 0x7fa7f412e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 -- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f409ba70 msgr2=0x7fa7f4135810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f409ba70 0x7fa7f4135810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.570+0000 7fa7fa800640 1 -- 172.21.15.99:0/339128497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f4135e80 con 0x7fa7f4096810 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa7fa800640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0349e0 tx=0x7fa7ec032450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec041030 con 0x7fa7f4096810 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec02c510 con 0x7fa7f4096810 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f4136110 con 0x7fa7f4096810 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec039040 con 0x7fa7f4096810 2024-03-23T23:40:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.571+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4136600 con 0x7fa7f4096810 2024-03-23T23:40:55.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.573+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f4136860 con 0x7fa7f4096810 2024-03-23T23:40:55.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.575+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa7ec044070 con 0x7fa7f4096810 2024-03-23T23:40:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.577+0000 7fa7f3400640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 0x7fa7e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.577+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7ec0efbc0 con 0x7fa7f4096810 2024-03-23T23:40:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.577+0000 7fa7f9e00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 0x7fa7e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.578+0000 7fa7f9e00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 0x7fa7e00a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7f412f770 tx=0x7fa7e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:55.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.581+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7ec0bd060 con 0x7fa7f4096810 2024-03-23T23:40:55.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.837+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f4136a30 con 0x7fa7f4096810 2024-03-23T23:40:55.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.838+0000 7fa7f3400640 1 -- 172.21.15.99:0/339128497 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7fa7ec0bc960 con 0x7fa7f4096810 2024-03-23T23:40:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:55.839 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-23T23:40:55.839 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 10 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 msgr2=0x7fa7e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 0x7fa7e00a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7f412f770 tx=0x7fa7e8009260 comp rx=0 tx=0).stop 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 msgr2=0x7fa7f412e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0349e0 tx=0x7fa7ec032450 comp rx=0 tx=0).stop 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 shutdown_connections 2024-03-23T23:40:55.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa7e00a2800 0x7fa7e00a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f409ba70 0x7fa7f4135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7f4099140 0x7fa7f412e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 --2- 172.21.15.99:0/339128497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7f4096810 0x7fa7f412e320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 >> 172.21.15.99:0/339128497 conn(0x7fa7f40901e0 msgr2=0x7fa7f4097680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 shutdown_connections 2024-03-23T23:40:55.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:55.841+0000 7fa800a00640 1 -- 172.21.15.99:0/339128497 wait complete. 2024-03-23T23:40:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:55 smithi099 ceph-mon[31715]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:55 smithi149 ceph-mon[36083]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:55 smithi174 ceph-mon[37002]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:56.369 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 podman[70822]: 2024-03-23 23:40:56.197927433 +0000 UTC m=+0.179302144 container create 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:40:56.786 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 podman[70822]: 2024-03-23 23:40:56.518565495 +0000 UTC m=+0.499940261 container init 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:40:56.786 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 podman[70822]: 2024-03-23 23:40:56.527635649 +0000 UTC m=+0.509010378 container start 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-23T23:40:56.787 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mds[70842]: -- 172.21.15.99:0/349947577 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a2af77a40 con 0x558a2af4b000 2024-03-23T23:40:56.787 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: starting mds.g at 2024-03-23T23:40:56.787 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 bash[70822]: 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 2024-03-23T23:40:56.787 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:40:56 smithi099 systemd[1]: Started Ceph mds.g for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:40:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/339128497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:56 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:56 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/339128497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:56 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:56 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/339128497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:40:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:56 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:40:57.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:40:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 -- 172.21.15.99:0/173697501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4109890 msgr2=0x7fd6e4110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 --2- 172.21.15.99:0/173697501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4109890 0x7fd6e4110120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ca0 tx=0x7fd6dc02c100 comp rx=0 tx=0).stop 2024-03-23T23:40:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 -- 172.21.15.99:0/173697501 shutdown_connections 2024-03-23T23:40:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 --2- 172.21.15.99:0/173697501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4109890 0x7fd6e4110120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 --2- 172.21.15.99:0/173697501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 --2- 172.21.15.99:0/173697501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4107ce0 0x7fd6e4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 -- 172.21.15.99:0/173697501 >> 172.21.15.99:0/173697501 conn(0x7fd6e4076cb0 msgr2=0x7fd6e40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 -- 172.21.15.99:0/173697501 shutdown_connections 2024-03-23T23:40:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.070+0000 7fd6eae00640 1 -- 172.21.15.99:0/173697501 wait complete. 2024-03-23T23:40:58.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:58.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:58.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:58.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: Deploying daemon mds.a on smithi099 2024-03-23T23:40:58.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:boot 2024-03-23T23:40:58.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: fsmap 10 up:standby 2024-03-23T23:40:58.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.073 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:57 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 Processor -- start 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 -- start start 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4109890 0x7fd6e41a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4113940 con 0x7fd6e4107ce0 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41137c0 con 0x7fd6e4108ed0 2024-03-23T23:40:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6eae00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4113ac0 con 0x7fd6e4109890 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.072+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56782/0 (socket says 172.21.15.99:56782) 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 -- 172.21.15.99:0/2973637615 learned_addr learned my addr 172.21.15.99:0/2973637615 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6db400640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e8e00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4109890 0x7fd6e41a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 -- 172.21.15.99:0/2973637615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4109890 msgr2=0x7fd6e41a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4109890 0x7fd6e41a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 -- 172.21.15.99:0/2973637615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 msgr2=0x7fd6e419e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e419e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 -- 172.21.15.99:0/2973637615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e41a5ba0 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6db400640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e419e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e3e00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9c0 tx=0x7fd6d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400cd80 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6d40035f0 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.073+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4010690 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.074+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e41a5e90 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.074+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e41a6350 con 0x7fd6e4107ce0 2024-03-23T23:40:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.075+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e410f080 con 0x7fd6e4107ce0 2024-03-23T23:40:58.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.077+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd6d40037a0 con 0x7fd6e4107ce0 2024-03-23T23:40:58.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.079+0000 7fd6e1600640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 0x7fd6c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:40:58.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.079+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6d40ceb20 con 0x7fd6e4107ce0 2024-03-23T23:40:58.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.079+0000 7fd6db400640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 0x7fd6c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:40:58.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.080+0000 7fd6db400640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 0x7fd6c80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6e419f490 tx=0x7fd6cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:40:58.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.084+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd6d409c0f0 con 0x7fd6e4107ce0 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: Deploying daemon mds.a on smithi099 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:boot 2024-03-23T23:40:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: fsmap 10 up:standby 2024-03-23T23:40:58.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:57 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: Deploying daemon mds.a on smithi099 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:boot 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: fsmap 10 up:standby 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:57 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:40:58.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.326+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e402d070 con 0x7fd6e4107ce0 2024-03-23T23:40:58.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.328+0000 7fd6e1600640 1 -- 172.21.15.99:0/2973637615 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7fd6d409b9f0 con 0x7fd6e4107ce0 2024-03-23T23:40:58.331 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:40:58.331 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 11 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 msgr2=0x7fd6c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 0x7fd6c80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6e419f490 tx=0x7fd6cc005f90 comp rx=0 tx=0).stop 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 msgr2=0x7fd6e419df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e9c0 tx=0x7fd6d400ee80 comp rx=0 tx=0).stop 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 shutdown_connections 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6c80a2880 0x7fd6c80a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e4109890 0x7fd6e41a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6e4108ed0 0x7fd6e419e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 --2- 172.21.15.99:0/2973637615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6e4107ce0 0x7fd6e419df80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:40:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 >> 172.21.15.99:0/2973637615 conn(0x7fd6e4076cb0 msgr2=0x7fd6e410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:40:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 shutdown_connections 2024-03-23T23:40:58.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:40:58.331+0000 7fd6eae00640 1 -- 172.21.15.99:0/2973637615 wait complete. 2024-03-23T23:40:58.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:58 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2973637615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:59.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:58 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2973637615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:59.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:58 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2973637615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:40:59.849 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:40:59 smithi099 ceph-mon[31715]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:00.111 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:40:59 smithi099 systemd[1]: Starting Ceph mds.a for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:41:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:40:59 smithi149 ceph-mon[36083]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:00.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:40:59 smithi174 ceph-mon[37002]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:00.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:41:00.516 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 podman[71163]: 2024-03-23 23:41:00.104916136 +0000 UTC m=+0.033816250 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:41:00.517 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 podman[71163]: 2024-03-23 23:41:00.237690171 +0000 UTC m=+0.166590292 container create beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.564+0000 7fc72b400640 1 -- 172.21.15.99:0/2139544867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7241038e0 msgr2=0x7fc724103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.564+0000 7fc72b400640 1 --2- 172.21.15.99:0/2139544867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7241038e0 0x7fc724103d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.564+0000 7fc72b400640 1 -- 172.21.15.99:0/2139544867 shutdown_connections 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.564+0000 7fc72b400640 1 --2- 172.21.15.99:0/2139544867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724105490 0x7fc72410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.565+0000 7fc72b400640 1 --2- 172.21.15.99:0/2139544867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc724104ad0 0x7fc724104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.565+0000 7fc72b400640 1 --2- 172.21.15.99:0/2139544867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7241038e0 0x7fc724103d00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.565+0000 7fc72b400640 1 -- 172.21.15.99:0/2139544867 >> 172.21.15.99:0/2139544867 conn(0x7fc7240fee50 msgr2=0x7fc724101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.565+0000 7fc72b400640 1 -- 172.21.15.99:0/2139544867 shutdown_connections 2024-03-23T23:41:00.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.565+0000 7fc72b400640 1 -- 172.21.15.99:0/2139544867 wait complete. 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 Processor -- start 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 -- start start 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724104ad0 0x7fc72419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724105490 0x7fc7241a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72410f5a0 con 0x7fc724105490 2024-03-23T23:41:00.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72410f420 con 0x7fc724104ad0 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.566+0000 7fc72b400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72410f720 con 0x7fc7241038e0 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52216/0 (socket says 172.21.15.99:52216) 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 -- 172.21.15.99:0/2676161227 learned_addr learned my addr 172.21.15.99:0/2676161227 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc723e00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724104ad0 0x7fc72419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc729400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724105490 0x7fc7241a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 -- 172.21.15.99:0/2676161227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724104ad0 msgr2=0x7fc72419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724104ad0 0x7fc72419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 -- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724105490 msgr2=0x7fc7241a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724105490 0x7fc7241a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 -- 172.21.15.99:0/2676161227 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7241a1870 con 0x7fc7241038e0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.567+0000 7fc728a00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc718000c00 tx=0x7fc71802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.568+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718043020 con 0x7fc7241038e0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.568+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7180054e0 con 0x7fc7241038e0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.568+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718003c40 con 0x7fc7241038e0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.568+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7241a1b00 con 0x7fc7241038e0 2024-03-23T23:41:00.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.568+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7241a2000 con 0x7fc7241038e0 2024-03-23T23:41:00.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.569+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240688e0 con 0x7fc7241038e0 2024-03-23T23:41:00.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.571+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc718046020 con 0x7fc7241038e0 2024-03-23T23:41:00.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.572+0000 7fc721600640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 0x7fc70c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:00.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.572+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7180f0650 con 0x7fc7241038e0 2024-03-23T23:41:00.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.572+0000 7fc723e00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 0x7fc70c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:00.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.573+0000 7fc723e00640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 0x7fc70c0a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc710009bd0 tx=0x7fc710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:00.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.576+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc71804d020 con 0x7fc7241038e0 2024-03-23T23:41:00.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.837+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc72410ac00 con 0x7fc7241038e0 2024-03-23T23:41:00.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.838+0000 7fc721600640 1 -- 172.21.15.99:0/2676161227 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7fc7180bdaf0 con 0x7fc7241038e0 2024-03-23T23:41:00.838 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:00.839 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-23T23:41:00.839 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 12 2024-03-23T23:41:00.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 msgr2=0x7fc70c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 0x7fc70c0a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc710009bd0 tx=0x7fc710005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 msgr2=0x7fc724199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc718000c00 tx=0x7fc71802c650 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 shutdown_connections 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc70c0a2880 0x7fc70c0a4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724105490 0x7fc7241a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc724104ad0 0x7fc72419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 --2- 172.21.15.99:0/2676161227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7241038e0 0x7fc724199c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 >> 172.21.15.99:0/2676161227 conn(0x7fc7240fee50 msgr2=0x7fc724107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 shutdown_connections 2024-03-23T23:41:00.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:00.841+0000 7fc72b400640 1 -- 172.21.15.99:0/2676161227 wait complete. 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 podman[71163]: 2024-03-23 23:41:00.516453353 +0000 UTC m=+0.445353514 container init beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 podman[71163]: 2024-03-23 23:41:00.525817742 +0000 UTC m=+0.454717837 container start beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 ceph-mds[71201]: -- 172.21.15.99:0/1673772096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56519a18ba40 con 0x56519a15e800 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: starting mds.a at 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 bash[71163]: beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 2024-03-23T23:41:00.858 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:41:00 smithi099 systemd[1]: Started Ceph mds.a for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:boot 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: fsmap 11 up:standby 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:41:02.113 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:41:02.113 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:02.113 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: Deploying daemon mds.b on smithi149 2024-03-23T23:41:02.113 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:01 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2676161227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:02.128 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:02.128 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:boot 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: fsmap 11 up:standby 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: Deploying daemon mds.b on smithi149 2024-03-23T23:41:02.129 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:01 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2676161227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:boot 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: fsmap 11 up:standby 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:41:02.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:02.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T23:41:02.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T23:41:02.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:02.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: Deploying daemon mds.b on smithi149 2024-03-23T23:41:02.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:01 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2676161227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:02.862 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:41:03.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.083+0000 7f5f22600640 1 -- 172.21.15.99:0/467528894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c108ed0 msgr2=0x7f5f1c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:03.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.083+0000 7f5f22600640 1 --2- 172.21.15.99:0/467528894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c109350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009900 tx=0x7f5f0402bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:03.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 -- 172.21.15.99:0/467528894 shutdown_connections 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 --2- 172.21.15.99:0/467528894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c109890 0x7f5f1c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 --2- 172.21.15.99:0/467528894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c109350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 --2- 172.21.15.99:0/467528894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 -- 172.21.15.99:0/467528894 >> 172.21.15.99:0/467528894 conn(0x7f5f1c076cb0 msgr2=0x7f5f1c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 -- 172.21.15.99:0/467528894 shutdown_connections 2024-03-23T23:41:03.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.084+0000 7f5f22600640 1 -- 172.21.15.99:0/467528894 wait complete. 2024-03-23T23:41:03.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.085+0000 7f5f22600640 1 Processor -- start 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.085+0000 7f5f22600640 1 -- start start 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.085+0000 7f5f22600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f22600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f22600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f22600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1139b0 con 0x7f5f1c109890 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f22600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c113830 con 0x7f5f1c107ce0 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f22600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c113b30 con 0x7f5f1c108ed0 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39030/0 (socket says 172.21.15.99:39030) 2024-03-23T23:41:03.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 -- 172.21.15.99:0/3883361096 learned_addr learned my addr 172.21.15.99:0/3883361096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1b400640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f13e00640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 -- 172.21.15.99:0/3883361096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c108ed0 msgr2=0x7f5f1c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 -- 172.21.15.99:0/3883361096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 msgr2=0x7f5f1c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1be00640 1 -- 172.21.15.99:0/3883361096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c1a5b90 con 0x7f5f1c109890 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.086+0000 7f5f1b400640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f1be00640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400eb10 tx=0x7f5f1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14004b20 con 0x7f5f1c109890 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f14004cc0 con 0x7f5f1c109890 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1a5e80 con 0x7f5f1c109890 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14010760 con 0x7f5f1c109890 2024-03-23T23:41:03.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.087+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c1a6390 con 0x7f5f1c109890 2024-03-23T23:41:03.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.088+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c10efa0 con 0x7f5f1c109890 2024-03-23T23:41:03.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.090+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5f14005020 con 0x7f5f1c109890 2024-03-23T23:41:03.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.092+0000 7f5f18c00640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 0x7f5f000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:03.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.092+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5f140cec90 con 0x7f5f1c109890 2024-03-23T23:41:03.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.092+0000 7f5f1b400640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 0x7f5f000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:03.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.092+0000 7f5f1b400640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 0x7f5f000a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c108d30 tx=0x7f5f0c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:03.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.096+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f1409c130 con 0x7f5f1c109890 2024-03-23T23:41:03.106 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:02 smithi149 systemd[1]: Starting Ceph mds.b for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:41:03.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.331+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f1c02d070 con 0x7f5f1c109890 2024-03-23T23:41:03.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.332+0000 7f5f18c00640 1 -- 172.21.15.99:0/3883361096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f5f1409ba30 con 0x7f5f1c109890 2024-03-23T23:41:03.332 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:03.332 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-03-23T23:41:03.332 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 12 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 msgr2=0x7f5f000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 0x7f5f000a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c108d30 tx=0x7f5f0c00b390 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 msgr2=0x7f5f1c1a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400eb10 tx=0x7f5f1400c2d0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 shutdown_connections 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.334+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5f000a2880 0x7f5f000a4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f1c109890 0x7f5f1c1a5520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f1c108ed0 0x7f5f1c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 --2- 172.21.15.99:0/3883361096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f1c107ce0 0x7f5f1c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 >> 172.21.15.99:0/3883361096 conn(0x7f5f1c076cb0 msgr2=0x7f5f1c10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 shutdown_connections 2024-03-23T23:41:03.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:03.335+0000 7f5f22600640 1 -- 172.21.15.99:0/3883361096 wait complete. 2024-03-23T23:41:03.394 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 podman[66170]: 2024-03-23 23:41:03.107303121 +0000 UTC m=+0.114486761 container create 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-23T23:41:03.395 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 podman[66170]: 2024-03-23 23:41:03.010820645 +0000 UTC m=+0.018004256 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:41:03.395 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 podman[66170]: 2024-03-23 23:41:03.326776622 +0000 UTC m=+0.333960234 container init 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-23T23:41:03.395 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 podman[66170]: 2024-03-23 23:41:03.329317463 +0000 UTC m=+0.336501056 container start 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:41:03.395 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mds[66189]: -- 172.21.15.149:0/548043458 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fee495a40 con 0x558fee469000 2024-03-23T23:41:03.395 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: starting mds.b at 2024-03-23T23:41:03.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 bash[66170]: 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 2024-03-23T23:41:03.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:03 smithi149 systemd[1]: Started Ceph mds.b for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:41:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3883361096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:03 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3883361096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:03 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3883361096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:03 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:04.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:04 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:boot 2024-03-23T23:41:04.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:04 smithi149 ceph-mon[36083]: fsmap 12 up:standby 2024-03-23T23:41:04.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:04 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:41:04.894 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:04 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:04.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:04 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:boot 2024-03-23T23:41:04.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:04 smithi174 ceph-mon[37002]: fsmap 12 up:standby 2024-03-23T23:41:04.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:04 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:41:04.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:04 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:05.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:04 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:boot 2024-03-23T23:41:05.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:04 smithi099 ceph-mon[31715]: fsmap 12 up:standby 2024-03-23T23:41:05.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:04 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:41:05.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:04 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:05.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 -- 172.21.15.99:0/361400209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380109890 msgr2=0x7fd380110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 --2- 172.21.15.99:0/361400209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380109890 0x7fd380110120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd378009be0 tx=0x7fd37802bef0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 -- 172.21.15.99:0/361400209 shutdown_connections 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 --2- 172.21.15.99:0/361400209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380109890 0x7fd380110120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 --2- 172.21.15.99:0/361400209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd380109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 --2- 172.21.15.99:0/361400209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380107ce0 0x7fd380108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.564+0000 7fd386200640 1 -- 172.21.15.99:0/361400209 >> 172.21.15.99:0/361400209 conn(0x7fd380076cb0 msgr2=0x7fd3800790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.565+0000 7fd386200640 1 -- 172.21.15.99:0/361400209 shutdown_connections 2024-03-23T23:41:05.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.565+0000 7fd386200640 1 -- 172.21.15.99:0/361400209 wait complete. 2024-03-23T23:41:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.565+0000 7fd386200640 1 Processor -- start 2024-03-23T23:41:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 -- start start 2024-03-23T23:41:05.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380107ce0 0x7fd38019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd38019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380113990 con 0x7fd380108ed0 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380113810 con 0x7fd380109890 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd386200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380113b10 con 0x7fd380107ce0 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd377e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd38019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd377e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd38019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39044/0 (socket says 172.21.15.99:39044) 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:52562/0 (socket says 172.21.15.99:52562) 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd377e00640 1 -- 172.21.15.99:0/3831804115 learned_addr learned my addr 172.21.15.99:0/3831804115 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.566+0000 7fd37f400640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380107ce0 0x7fd38019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 -- 172.21.15.99:0/3831804115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380107ce0 msgr2=0x7fd38019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380107ce0 0x7fd38019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 -- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 msgr2=0x7fd38019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd38019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 -- 172.21.15.99:0/3831804115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3801a5b60 con 0x7fd380109890 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37fe00640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd37802c430 tx=0x7fd3780337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378043020 con 0x7fd380109890 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd37802ca60 con 0x7fd380109890 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3801a5df0 con 0x7fd380109890 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378034690 con 0x7fd380109890 2024-03-23T23:41:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.567+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801a62b0 con 0x7fd380109890 2024-03-23T23:41:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.568+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38010efa0 con 0x7fd380109890 2024-03-23T23:41:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.570+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd378046020 con 0x7fd380109890 2024-03-23T23:41:05.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.571+0000 7fd37cc00640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 0x7fd3640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:05.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.572+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd3780f0b90 con 0x7fd380109890 2024-03-23T23:41:05.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.572+0000 7fd37f400640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 0x7fd3640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:05.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.572+0000 7fd37f400640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 0x7fd3640a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd370009530 tx=0x7fd370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.575+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd37804d020 con 0x7fd380109890 2024-03-23T23:41:05.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.821+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd38002d470 con 0x7fd380109890 2024-03-23T23:41:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.822+0000 7fd37cc00640 1 -- 172.21.15.99:0/3831804115 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7fd3780be030 con 0x7fd380109890 2024-03-23T23:41:05.822 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:05.822 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-23T23:41:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 13 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.824+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 msgr2=0x7fd3640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.824+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 0x7fd3640a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd370009530 tx=0x7fd370009260 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.824+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 msgr2=0x7fd3801a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.824+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd37802c430 tx=0x7fd3780337b0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 shutdown_connections 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3640a2800 0x7fd3640a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd380109890 0x7fd3801a54f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd380108ed0 0x7fd38019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 --2- 172.21.15.99:0/3831804115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd380107ce0 0x7fd38019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 >> 172.21.15.99:0/3831804115 conn(0x7fd380076cb0 msgr2=0x7fd38010c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 shutdown_connections 2024-03-23T23:41:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:05.825+0000 7fd386200640 1 -- 172.21.15.99:0/3831804115 wait complete. 2024-03-23T23:41:05.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.033+0000 7ffa41800640 1 -- 172.21.15.99:0/3744898463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c105490 msgr2=0x7ffa3c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.033+0000 7ffa41800640 1 --2- 172.21.15.99:0/3744898463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c105490 0x7ffa3c10bd20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009ca0 tx=0x7ffa3402c100 comp rx=0 tx=0).stop 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 -- 172.21.15.99:0/3744898463 shutdown_connections 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 --2- 172.21.15.99:0/3744898463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c105490 0x7ffa3c10bd20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 --2- 172.21.15.99:0/3744898463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c104ad0 0x7ffa3c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 --2- 172.21.15.99:0/3744898463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c1038e0 0x7ffa3c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 -- 172.21.15.99:0/3744898463 >> 172.21.15.99:0/3744898463 conn(0x7ffa3c0fee90 msgr2=0x7ffa3c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 -- 172.21.15.99:0/3744898463 shutdown_connections 2024-03-23T23:41:06.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.034+0000 7ffa41800640 1 -- 172.21.15.99:0/3744898463 wait complete. 2024-03-23T23:41:06.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 Processor -- start 2024-03-23T23:41:06.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 -- start start 2024-03-23T23:41:06.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1038e0 0x7ffa3c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c104ad0 0x7ffa3c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c10f560 con 0x7ffa3c104ad0 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c10f3e0 con 0x7ffa3c105490 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa41800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c10f6e0 con 0x7ffa3c1038e0 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa3a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c104ad0 0x7ffa3c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:52576/0 (socket says 172.21.15.99:52576) 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa3b400640 1 -- 172.21.15.99:0/979217660 learned_addr learned my addr 172.21.15.99:0/979217660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.035+0000 7ffa3aa00640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1038e0 0x7ffa3c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 -- 172.21.15.99:0/979217660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1038e0 msgr2=0x7ffa3c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1038e0 0x7ffa3c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 -- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c104ad0 msgr2=0x7ffa3c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c104ad0 0x7ffa3c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 -- 172.21.15.99:0/979217660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c1a17c0 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa3b400640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa3402c640 tx=0x7ffa34035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34039040 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa34044070 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340043c0 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.036+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c1a1a50 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.037+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c1a1f40 con 0x7ffa3c105490 2024-03-23T23:41:06.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.038+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c10aba0 con 0x7ffa3c105490 2024-03-23T23:41:06.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.039+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffa34004560 con 0x7ffa3c105490 2024-03-23T23:41:06.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.040+0000 7ffa33400640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 0x7ffa200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.041+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffa340f0b70 con 0x7ffa3c105490 2024-03-23T23:41:06.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.041+0000 7ffa3aa00640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 0x7ffa200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.041+0000 7ffa3aa00640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 0x7ffa200a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c104930 tx=0x7ffa2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.044+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa340be0b0 con 0x7ffa3c105490 2024-03-23T23:41:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:05 smithi099 ceph-mon[31715]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:05 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:05 smithi149 ceph-mon[36083]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:05 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:05 smithi174 ceph-mon[37002]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:05 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.256+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa3c0622a0 con 0x7ffa3c105490 2024-03-23T23:41:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.257+0000 7ffa33400640 1 -- 172.21.15.99:0/979217660 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7ffa340098b0 con 0x7ffa3c105490 2024-03-23T23:41:06.257 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:06.258 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":81,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:40:12.704666+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 msgr2=0x7ffa200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 0x7ffa200a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c104930 tx=0x7ffa2c005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 msgr2=0x7ffa3c1a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa3402c640 tx=0x7ffa34035d10 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 shutdown_connections 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ffa200a2800 0x7ffa200a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa3c105490 0x7ffa3c1a1150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa3c104ad0 0x7ffa3c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 --2- 172.21.15.99:0/979217660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1038e0 0x7ffa3c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 >> 172.21.15.99:0/979217660 conn(0x7ffa3c0fee90 msgr2=0x7ffa3c107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 shutdown_connections 2024-03-23T23:41:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.260+0000 7ffa41800640 1 -- 172.21.15.99:0/979217660 wait complete. 2024-03-23T23:41:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:41:06.277 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:41:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 -- 172.21.15.99:0/4183064326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1981038e0 msgr2=0x7fc198103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 --2- 172.21.15.99:0/4183064326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1981038e0 0x7fc198103d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009900 tx=0x7fc18c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 -- 172.21.15.99:0/4183064326 shutdown_connections 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 --2- 172.21.15.99:0/4183064326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 0x7fc19810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 --2- 172.21.15.99:0/4183064326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc198104ad0 0x7fc198104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 --2- 172.21.15.99:0/4183064326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1981038e0 0x7fc198103d00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 -- 172.21.15.99:0/4183064326 >> 172.21.15.99:0/4183064326 conn(0x7fc1980fee50 msgr2=0x7fc198101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 -- 172.21.15.99:0/4183064326 shutdown_connections 2024-03-23T23:41:06.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.467+0000 7fc1a0800640 1 -- 172.21.15.99:0/4183064326 wait complete. 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.468+0000 7fc1a0800640 1 Processor -- start 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.468+0000 7fc1a0800640 1 -- start start 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 0x7fc19819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 0x7fc1981a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19810f580 con 0x7fc1981038e0 2024-03-23T23:41:06.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19810f400 con 0x7fc198104ad0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc1a0800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19810f700 con 0x7fc198105490 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39082/0 (socket says 172.21.15.99:39082) 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19de00640 1 -- 172.21.15.99:0/1594540206 learned_addr learned my addr 172.21.15.99:0/1594540206 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19d400640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 0x7fc19819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19e800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 0x7fc1981a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19d400640 1 -- 172.21.15.99:0/1594540206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 msgr2=0x7fc1981a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.469+0000 7fc19d400640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 0x7fc1981a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc19d400640 1 -- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 msgr2=0x7fc198199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc19d400640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc19d400640 1 -- 172.21.15.99:0/1594540206 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1981a1850 con 0x7fc198104ad0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc19de00640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc19d400640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 0x7fc19819a180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc18800e9f0 tx=0x7fc18800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800ccc0 con 0x7fc198104ad0 2024-03-23T23:41:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc188010910 con 0x7fc198104ad0 2024-03-23T23:41:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981a1b40 con 0x7fc198104ad0 2024-03-23T23:41:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188018960 con 0x7fc198104ad0 2024-03-23T23:41:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.470+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981a20e0 con 0x7fc198104ad0 2024-03-23T23:41:06.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.471+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19810ac50 con 0x7fc198104ad0 2024-03-23T23:41:06.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.473+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc188010ac0 con 0x7fc198104ad0 2024-03-23T23:41:06.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.475+0000 7fc192a00640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.475+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1880ce9e0 con 0x7fc198104ad0 2024-03-23T23:41:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.475+0000 7fc19de00640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.476+0000 7fc19de00640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 0x7fc1800a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc19819b110 tx=0x7fc18c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.478+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc18809b830 con 0x7fc198104ad0 2024-03-23T23:41:06.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.697+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1980622a0 con 0x7fc198104ad0 2024-03-23T23:41:06.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.697+0000 7fc192a00640 1 -- 172.21.15.99:0/1594540206 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc18809f8d0 con 0x7fc198104ad0 2024-03-23T23:41:06.698 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 msgr2=0x7fc1800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 0x7fc1800a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc19819b110 tx=0x7fc18c02df90 comp rx=0 tx=0).stop 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 msgr2=0x7fc19819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 0x7fc19819a180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc18800e9f0 tx=0x7fc18800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 shutdown_connections 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc1800a2880 0x7fc1800a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc198105490 0x7fc1981a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc198104ad0 0x7fc19819a180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 --2- 172.21.15.99:0/1594540206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1981038e0 0x7fc198199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 >> 172.21.15.99:0/1594540206 conn(0x7fc1980fee50 msgr2=0x7fc198107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:06.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 shutdown_connections 2024-03-23T23:41:06.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.700+0000 7fc1a0800640 1 -- 172.21.15.99:0/1594540206 wait complete. 2024-03-23T23:41:06.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 -- 172.21.15.99:0/555730612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614107b00 msgr2=0x7fa614107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 --2- 172.21.15.99:0/555730612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614107b00 0x7fa614107f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60402bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 -- 172.21.15.99:0/555730612 shutdown_connections 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 --2- 172.21.15.99:0/555730612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa61410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 --2- 172.21.15.99:0/555730612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614108cf0 0x7fa614109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 --2- 172.21.15.99:0/555730612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614107b00 0x7fa614107f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 -- 172.21.15.99:0/555730612 >> 172.21.15.99:0/555730612 conn(0x7fa614076ab0 msgr2=0x7fa614078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 -- 172.21.15.99:0/555730612 shutdown_connections 2024-03-23T23:41:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.892+0000 7fa61a200640 1 -- 172.21.15.99:0/555730612 wait complete. 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 Processor -- start 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 -- start start 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614107b00 0x7fa61419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa6141a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6141137c0 con 0x7fa614108cf0 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614113640 con 0x7fa6141096b0 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa61a200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614113940 con 0x7fa614107b00 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa60be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa60be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39106/0 (socket says 172.21.15.99:39106) 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa60be00640 1 -- 172.21.15.99:0/1230382990 learned_addr learned my addr 172.21.15.99:0/1230382990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa618e00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614107b00 0x7fa61419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.893+0000 7fa619800640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa6141a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 -- 172.21.15.99:0/1230382990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614107b00 msgr2=0x7fa61419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614107b00 0x7fa61419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 -- 172.21.15.99:0/1230382990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 msgr2=0x7fa6141a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa6141a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 -- 172.21.15.99:0/1230382990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6141a5a50 con 0x7fa614108cf0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa619800640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa6141a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa60be00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9f0 tx=0x7fa5fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00cd80 con 0x7fa614108cf0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5fc005500 con 0x7fa614108cf0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6141a5d40 con 0x7fa614108cf0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc010690 con 0x7fa614108cf0 2024-03-23T23:41:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.894+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6141a6230 con 0x7fa614108cf0 2024-03-23T23:41:06.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.896+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61410ee20 con 0x7fa614108cf0 2024-03-23T23:41:06.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.897+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa5fc0046c0 con 0x7fa614108cf0 2024-03-23T23:41:06.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.899+0000 7fa612000640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 0x7fa5f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:06.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.899+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5fc0ceb40 con 0x7fa614108cf0 2024-03-23T23:41:06.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.899+0000 7fa618e00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 0x7fa5f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:06.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.900+0000 7fa618e00640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 0x7fa5f80a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa60402c170 tx=0x7fa60402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:06.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:06.903+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5fc09bfe0 con 0x7fa614108cf0 2024-03-23T23:41:07.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.104+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa614062340 con 0x7fa614108cf0 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.105+0000 7fa612000640 1 -- 172.21.15.99:0/1230382990 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa5fc09b8e0 con 0x7fa614108cf0 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:41:07.105 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 msgr2=0x7fa5f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 0x7fa5f80a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa60402c170 tx=0x7fa60402df90 comp rx=0 tx=0).stop 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 msgr2=0x7fa61419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e9f0 tx=0x7fa5fc00edf0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 shutdown_connections 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5f80a2880 0x7fa5f80a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6141096b0 0x7fa6141a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa614108cf0 0x7fa61419e370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 --2- 172.21.15.99:0/1230382990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614107b00 0x7fa61419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 >> 172.21.15.99:0/1230382990 conn(0x7fa614076ab0 msgr2=0x7fa61410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:07.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 shutdown_connections 2024-03-23T23:41:07.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.107+0000 7fa61a200640 1 -- 172.21.15.99:0/1230382990 wait complete. 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3831804115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/979217660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1594540206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:41:07.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.125 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T23:41:07.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3831804115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/979217660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1594540206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:41:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3831804115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/979217660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1594540206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:41:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:07.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 -- 172.21.15.99:0/3242491067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72500fff20 msgr2=0x7f7250102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:07.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 --2- 172.21.15.99:0/3242491067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72500fff20 0x7f7250102350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7240009900 tx=0x7f724002bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:07.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 -- 172.21.15.99:0/3242491067 shutdown_connections 2024-03-23T23:41:07.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 --2- 172.21.15.99:0/3242491067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 0x7f72501056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 --2- 172.21.15.99:0/3242491067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7250102890 0x7f7250104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 --2- 172.21.15.99:0/3242491067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72500fff20 0x7f7250102350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 -- 172.21.15.99:0/3242491067 >> 172.21.15.99:0/3242491067 conn(0x7f72500fbb40 msgr2=0x7f72500fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:07.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 -- 172.21.15.99:0/3242491067 shutdown_connections 2024-03-23T23:41:07.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.327+0000 7f7255800640 1 -- 172.21.15.99:0/3242491067 wait complete. 2024-03-23T23:41:07.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 Processor -- start 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 -- start start 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72500fff20 0x7f725019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 0x7f72501a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250113970 con 0x7f72500fff20 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501137f0 con 0x7f7250105240 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f7255800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250113af0 con 0x7f7250102890 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f724e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f724ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72500fff20 0x7f725019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f724e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52330/0 (socket says 172.21.15.99:52330) 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f724e000640 1 -- 172.21.15.99:0/2422868595 learned_addr learned my addr 172.21.15.99:0/2422868595 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:07.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.328+0000 7f724f400640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 0x7f72501a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 -- 172.21.15.99:0/2422868595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 msgr2=0x7f72501a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 0x7f72501a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 -- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72500fff20 msgr2=0x7f725019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72500fff20 0x7f725019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 -- 172.21.15.99:0/2422868595 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72501a5c30 con 0x7f7250102890 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f724e000640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f723c00e9f0 tx=0x7f723c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00cd80 con 0x7f7250102890 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72501a5f20 con 0x7f7250102890 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f723c010910 con 0x7f7250102890 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0188f0 con 0x7f7250102890 2024-03-23T23:41:07.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.329+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72501a6430 con 0x7f7250102890 2024-03-23T23:41:07.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.330+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72500fd630 con 0x7f7250102890 2024-03-23T23:41:07.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.333+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f723c010ac0 con 0x7f7250102890 2024-03-23T23:41:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.334+0000 7f7247400640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 0x7f72340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.334+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f723c0ced20 con 0x7f7250102890 2024-03-23T23:41:07.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.335+0000 7f724ea00640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 0x7f72340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:07.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.335+0000 7f724ea00640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 0x7f72340a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f724002c240 tx=0x7f724002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:07.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.336+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f723c09bb70 con 0x7f7250102890 2024-03-23T23:41:07.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:07.542+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f725002d110 con 0x7f7250102890 2024-03-23T23:41:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-03-23T23:41:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.160+0000 7f7247400640 1 -- 172.21.15.99:0/2422868595 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f723c09fc10 con 0x7f7250102890 2024-03-23T23:41:08.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.166+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 msgr2=0x7f72340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:08.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.166+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 0x7f72340a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f724002c240 tx=0x7f724002df90 comp rx=0 tx=0).stop 2024-03-23T23:41:08.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.166+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 msgr2=0x7f725019e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:08.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.166+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f723c00e9f0 tx=0x7f723c00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 shutdown_connections 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f72340a2880 0x7f72340a4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7250105240 0x7f72501a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7250102890 0x7f725019e550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 --2- 172.21.15.99:0/2422868595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72500fff20 0x7f725019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 >> 172.21.15.99:0/2422868595 conn(0x7f72500fbb40 msgr2=0x7f7250103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 shutdown_connections 2024-03-23T23:41:08.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.167+0000 7f7255800640 1 -- 172.21.15.99:0/2422868595 wait complete. 2024-03-23T23:41:08.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:08 smithi099 ceph-mon[31715]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:08.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:08 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:08.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:08 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1230382990' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T23:41:08.196 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T23:41:08.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 -- 172.21.15.99:0/628106237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa100772a0 msgr2=0x7ffa10077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:08.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 --2- 172.21.15.99:0/628106237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa100772a0 0x7ffa10077700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009900 tx=0x7ffa0002bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 -- 172.21.15.99:0/628106237 shutdown_connections 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 --2- 172.21.15.99:0/628106237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa10077c40 0x7ffa100784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 --2- 172.21.15.99:0/628106237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa100772a0 0x7ffa10077700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 --2- 172.21.15.99:0/628106237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa10076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 -- 172.21.15.99:0/628106237 >> 172.21.15.99:0/628106237 conn(0x7ffa100fee50 msgr2=0x7ffa10101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 -- 172.21.15.99:0/628106237 shutdown_connections 2024-03-23T23:41:08.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.385+0000 7ffa18600640 1 -- 172.21.15.99:0/628106237 wait complete. 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.386+0000 7ffa18600640 1 Processor -- start 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.386+0000 7ffa18600640 1 -- start start 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.386+0000 7ffa18600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.386+0000 7ffa18600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa100772a0 0x7ffa1019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.386+0000 7ffa18600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 0x7ffa101a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa18600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10113970 con 0x7ffa100772a0 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa18600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa101137f0 con 0x7ffa10077c40 2024-03-23T23:41:08.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa18600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10113af0 con 0x7ffa10076060 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa100772a0 0x7ffa1019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa16600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 0x7ffa101a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52340/0 (socket says 172.21.15.99:52340) 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 -- 172.21.15.99:0/3775760980 learned_addr learned my addr 172.21.15.99:0/3775760980 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 -- 172.21.15.99:0/3775760980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 msgr2=0x7ffa101a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 0x7ffa101a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 -- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa100772a0 msgr2=0x7ffa1019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa100772a0 0x7ffa1019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 -- 172.21.15.99:0/3775760980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa101a5c10 con 0x7ffa10076060 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa16600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 0x7ffa101a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.387+0000 7ffa15c00640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400ea60 tx=0x7ffa0400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.388+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400cde0 con 0x7ffa10076060 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.388+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa040035f0 con 0x7ffa10076060 2024-03-23T23:41:08.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.388+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa101a5f00 con 0x7ffa10076060 2024-03-23T23:41:08.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.388+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04010670 con 0x7ffa10076060 2024-03-23T23:41:08.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.388+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101a6360 con 0x7ffa10076060 2024-03-23T23:41:08.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.389+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa10110250 con 0x7ffa10076060 2024-03-23T23:41:08.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.391+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffa040037a0 con 0x7ffa10076060 2024-03-23T23:41:08.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.392+0000 7ffa0e000640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 0x7ff9f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:08.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.392+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7ffa040cea90 con 0x7ffa10076060 2024-03-23T23:41:08.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.392+0000 7ffa15200640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 0x7ff9f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:08.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.393+0000 7ffa15200640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 0x7ff9f80a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002c240 tx=0x7ffa0002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:08.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:08 smithi149 ceph-mon[36083]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:08.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:08 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:08.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:08 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1230382990' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T23:41:08.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.396+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa0409bdc0 con 0x7ffa10076060 2024-03-23T23:41:08.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:08 smithi174 ceph-mon[37002]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:08.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:08 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:41:08.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:08 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1230382990' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T23:41:08.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:08.598+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ffa1002d070 con 0x7ffa10076060 2024-03-23T23:41:09.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.236+0000 7ffa0e000640 1 -- 172.21.15.99:0/3775760980 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7ffa0409b6c0 con 0x7ffa10076060 2024-03-23T23:41:09.237 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 msgr2=0x7ff9f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 0x7ff9f80a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002c240 tx=0x7ffa0002df90 comp rx=0 tx=0).stop 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 msgr2=0x7ffa1019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400ea60 tx=0x7ffa0400ef20 comp rx=0 tx=0).stop 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 shutdown_connections 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9f80a2800 0x7ff9f80a4cc0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa0002c240 tx=0x7ffa0002df90 comp rx=0 tx=0).stop 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa10077c40 0x7ffa101a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa100772a0 0x7ffa1019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 --2- 172.21.15.99:0/3775760980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa10076060 0x7ffa1019dff0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400ea60 tx=0x7ffa0400ef20 comp rx=0 tx=0).stop 2024-03-23T23:41:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.243+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 >> 172.21.15.99:0/3775760980 conn(0x7ffa100fee50 msgr2=0x7ffa10100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.244+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 shutdown_connections 2024-03-23T23:41:09.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.244+0000 7ffa18600640 1 -- 172.21.15.99:0/3775760980 wait complete. 2024-03-23T23:41:09.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 -- 172.21.15.99:0/3622773621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 msgr2=0x7f5c58104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3622773621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58104570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009900 tx=0x7f5c4c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 -- 172.21.15.99:0/3622773621 shutdown_connections 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3622773621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c581073e0 0x7f5c581097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3622773621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58104ab0 0x7f5c58106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3622773621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58104570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.462+0000 7f5c5f400640 1 -- 172.21.15.99:0/3622773621 >> 172.21.15.99:0/3622773621 conn(0x7f5c580fbb30 msgr2=0x7f5c580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.463+0000 7f5c5f400640 1 -- 172.21.15.99:0/3622773621 shutdown_connections 2024-03-23T23:41:09.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.463+0000 7f5c5f400640 1 -- 172.21.15.99:0/3622773621 wait complete. 2024-03-23T23:41:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.463+0000 7f5c5f400640 1 Processor -- start 2024-03-23T23:41:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 -- start start 2024-03-23T23:41:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c58104ab0 0x7f5c58197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:09.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c581073e0 0x7f5c5819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5810d3b0 con 0x7f5c581073e0 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5810d230 con 0x7f5c58102180 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5f400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5810d530 con 0x7f5c58104ab0 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40624/0 (socket says 172.21.15.99:40624) 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c581073e0 0x7f5c5819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 -- 172.21.15.99:0/3320860581 learned_addr learned my addr 172.21.15.99:0/3320860581 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c57e00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c58104ab0 0x7f5c58197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 -- 172.21.15.99:0/3320860581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c58104ab0 msgr2=0x7f5c58197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c58104ab0 0x7f5c58197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 -- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c581073e0 msgr2=0x7f5c5819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c581073e0 0x7f5c5819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.464+0000 7f5c5ca00640 1 -- 172.21.15.99:0/3320860581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5819f680 con 0x7f5c58102180 2024-03-23T23:41:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c5ca00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02c8e0 tx=0x7f5c4c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c043020 con 0x7f5c58102180 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c0054e0 con 0x7f5c58102180 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5819f910 con 0x7f5c58102180 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c003c30 con 0x7f5c58102180 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.465+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5819fe60 con 0x7f5c58102180 2024-03-23T23:41:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.466+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58067c00 con 0x7f5c58102180 2024-03-23T23:41:09.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.469+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5c4c046020 con 0x7f5c58102180 2024-03-23T23:41:09.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.470+0000 7f5c55600640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 0x7f5c400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:09.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.470+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f5c4c0f0e30 con 0x7f5c58102180 2024-03-23T23:41:09.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.471+0000 7f5c57e00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 0x7f5c400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:09.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.471+0000 7f5c57e00640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 0x7f5c400a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c58198f70 tx=0x7f5c44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:09.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.472+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c4c04d020 con 0x7f5c58102180 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2422868595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3775760980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:09.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:09 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2422868595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3775760980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:09.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:09 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2422868595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3775760980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:09.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:09 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:09.725+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5c580fd620 con 0x7f5c58102180 2024-03-23T23:41:10.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: pgmap v313: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3320860581' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.612 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:10 smithi099 ceph-mon[31715]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: pgmap v313: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T23:41:10.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3320860581' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:10 smithi149 ceph-mon[36083]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T23:41:10.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.659+0000 7f5c55600640 1 -- 172.21.15.99:0/3320860581 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5c580fd620 con 0x7f5c58102180 2024-03-23T23:41:10.660 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T23:41:10.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 msgr2=0x7f5c400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:10.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 0x7f5c400a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c58198f70 tx=0x7f5c44005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:10.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 msgr2=0x7f5c58197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:10.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02c8e0 tx=0x7f5c4c004970 comp rx=0 tx=0).stop 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 shutdown_connections 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5c400a2800 0x7f5c400a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c581073e0 0x7f5c5819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c58104ab0 0x7f5c58197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 --2- 172.21.15.99:0/3320860581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58102180 0x7f5c58197a60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.667+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 >> 172.21.15.99:0/3320860581 conn(0x7f5c580fbb30 msgr2=0x7f5c58102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.668+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 shutdown_connections 2024-03-23T23:41:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.669+0000 7f5c5f400640 1 -- 172.21.15.99:0/3320860581 wait complete. 2024-03-23T23:41:10.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: pgmap v313: 33 pgs: 12 creating+peering, 1 active+clean, 20 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3320860581' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T23:41:10.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:10 smithi174 ceph-mon[37002]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T23:41:10.690 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 -- 172.21.15.99:0/3724756251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc076060 msgr2=0x7f9adc076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/3724756251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc076060 0x7f9adc076480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009900 tx=0x7f9acc02bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 -- 172.21.15.99:0/3724756251 shutdown_connections 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/3724756251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc077c40 0x7f9adc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/3724756251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/3724756251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc076060 0x7f9adc076480 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.882+0000 7f9ae2c00640 1 -- 172.21.15.99:0/3724756251 >> 172.21.15.99:0/3724756251 conn(0x7f9adc0fee50 msgr2=0x7f9adc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.883+0000 7f9ae2c00640 1 -- 172.21.15.99:0/3724756251 shutdown_connections 2024-03-23T23:41:10.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.883+0000 7f9ae2c00640 1 -- 172.21.15.99:0/3724756251 wait complete. 2024-03-23T23:41:10.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.883+0000 7f9ae2c00640 1 Processor -- start 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 -- start start 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc076060 0x7f9adc19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1139b0 con 0x7f9adc077c40 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc113830 con 0x7f9adc0772a0 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae2c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc113b30 con 0x7f9adc076060 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56056/0 (socket says 172.21.15.99:56056) 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9ae0c00640 1 -- 172.21.15.99:0/1864022451 learned_addr learned my addr 172.21.15.99:0/1864022451 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9adb400640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.884+0000 7f9adbe00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc076060 0x7f9adc19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 -- 172.21.15.99:0/1864022451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc076060 msgr2=0x7f9adc19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:10.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc076060 0x7f9adc19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 -- 172.21.15.99:0/1864022451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 msgr2=0x7f9adc19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 -- 172.21.15.99:0/1864022451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc1a5bd0 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9adb400640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae0c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400eb10 tx=0x7f9ad400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4004b20 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ad4004cc0 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc1a5ec0 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4010760 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.885+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1a6380 con 0x7f9adc077c40 2024-03-23T23:41:10.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.886+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc1103a0 con 0x7f9adc077c40 2024-03-23T23:41:10.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.888+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9ad4005000 con 0x7f9adc077c40 2024-03-23T23:41:10.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.890+0000 7f9ad8c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 0x7f9ac00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:10.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.890+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9ad40cf3e0 con 0x7f9adc077c40 2024-03-23T23:41:10.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.890+0000 7f9adbe00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 0x7f9ac00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:10.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.890+0000 7f9adbe00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 0x7f9ac00a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02c240 tx=0x7f9acc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:10.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:10.892+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ad409c530 con 0x7f9adc077c40 2024-03-23T23:41:11.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.091+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9adc02d2c0 con 0x7f9adc077c40 2024-03-23T23:41:11.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.096+0000 7f9ad8c00640 1 -- 172.21.15.99:0/1864022451 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22245 (secure 0 0 0) 0x7f9ad409be30 con 0x7f9adc077c40 2024-03-23T23:41:11.096 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:11.097 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:41:09.728194+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 msgr2=0x7f9ac00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 0x7f9ac00a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02c240 tx=0x7f9acc02df90 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 msgr2=0x7f9adc1a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400eb10 tx=0x7f9ad400c2d0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 shutdown_connections 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ac00a2800 0x7f9ac00a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9adc077c40 0x7f9adc1a5560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9adc0772a0 0x7f9adc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 --2- 172.21.15.99:0/1864022451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9adc076060 0x7f9adc19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 >> 172.21.15.99:0/1864022451 conn(0x7f9adc0fee50 msgr2=0x7f9adc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:11.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 shutdown_connections 2024-03-23T23:41:11.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.099+0000 7f9ae2c00640 1 -- 172.21.15.99:0/1864022451 wait complete. 2024-03-23T23:41:11.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 -- 172.21.15.99:0/3237680006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac103090 msgr2=0x7faaac109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 --2- 172.21.15.99:0/3237680006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac103090 0x7faaac109920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009ca0 tx=0x7faaa402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 -- 172.21.15.99:0/3237680006 shutdown_connections 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 --2- 172.21.15.99:0/3237680006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac103090 0x7faaac109920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 --2- 172.21.15.99:0/3237680006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac1026d0 0x7faaac102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.329+0000 7faaab200640 1 --2- 172.21.15.99:0/3237680006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 0x7faaac101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.330+0000 7faaab200640 1 -- 172.21.15.99:0/3237680006 >> 172.21.15.99:0/3237680006 conn(0x7faaac0fca30 msgr2=0x7faaac0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.330+0000 7faaab200640 1 -- 172.21.15.99:0/3237680006 shutdown_connections 2024-03-23T23:41:11.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.330+0000 7faaab200640 1 -- 172.21.15.99:0/3237680006 wait complete. 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 Processor -- start 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 -- start start 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 0x7faaac075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac103090 0x7faaac073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10d170 con 0x7faaac1026d0 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10cff0 con 0x7faaac103090 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaab200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10d2f0 con 0x7faaac1014e0 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaa9400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaa9400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56068/0 (socket says 172.21.15.99:56068) 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.331+0000 7faaa9400640 1 -- 172.21.15.99:0/3932538955 learned_addr learned my addr 172.21.15.99:0/3932538955 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9e00640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 0x7faaac075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaaa800640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac103090 0x7faaac073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 -- 172.21.15.99:0/3932538955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 msgr2=0x7faaac075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 0x7faaac075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 -- 172.21.15.99:0/3932538955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac103090 msgr2=0x7faaac073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac103090 0x7faaac073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 -- 172.21.15.99:0/3932538955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac074860 con 0x7faaac1026d0 2024-03-23T23:41:11.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa9400640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa9800e9f0 tx=0x7faa9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9800cd80 con 0x7faaac1026d0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac1a18c0 con 0x7faaac1026d0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa98010910 con 0x7faaac1026d0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.332+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98018950 con 0x7faaac1026d0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.333+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac1a1e10 con 0x7faaac1026d0 2024-03-23T23:41:11.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.334+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac108800 con 0x7faaac1026d0 2024-03-23T23:41:11.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.336+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faa98010ac0 con 0x7faaac1026d0 2024-03-23T23:41:11.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.338+0000 7faaa2a00640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 0x7faa900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.338+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa980cf0b0 con 0x7faaac1026d0 2024-03-23T23:41:11.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.338+0000 7faaa9e00640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 0x7faa900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.339+0000 7faaa9e00640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 0x7faa900a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0095f0 tx=0x7faa9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:11.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.340+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa9809bc50 con 0x7faaac1026d0 2024-03-23T23:41:11.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.541+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaac02d2e0 con 0x7faaac1026d0 2024-03-23T23:41:11.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.545+0000 7faaa2a00640 1 -- 172.21.15.99:0/3932538955 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7faa9809fcf0 con 0x7faaac1026d0 2024-03-23T23:41:11.545 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:11.545 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:41:11.233681+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.547+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 msgr2=0x7faa900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.547+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 0x7faa900a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0095f0 tx=0x7faa9c005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.547+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 msgr2=0x7faaac073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.547+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa9800e9f0 tx=0x7faa9800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.547+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 shutdown_connections 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa900a2800 0x7faa900a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac103090 0x7faaac073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaac1026d0 0x7faaac073a90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 --2- 172.21.15.99:0/3932538955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faaac1014e0 0x7faaac075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 >> 172.21.15.99:0/3932538955 conn(0x7faaac0fca30 msgr2=0x7faaac1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 shutdown_connections 2024-03-23T23:41:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.548+0000 7faaab200640 1 -- 172.21.15.99:0/3932538955 wait complete. 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: pgmap v316: 97 pgs: 32 active+clean, 24 creating+peering, 41 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:11.563 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:11 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1864022451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:11.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: pgmap v316: 97 pgs: 32 active+clean, 24 creating+peering, 41 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:11.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:11 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1864022451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: pgmap v316: 97 pgs: 32 active+clean, 24 creating+peering, 41 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:41:11.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:11 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1864022451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 -- 172.21.15.99:0/81270377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 msgr2=0x7f5810077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 --2- 172.21.15.99:0/81270377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f5810077700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009900 tx=0x7f57f802bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 -- 172.21.15.99:0/81270377 shutdown_connections 2024-03-23T23:41:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 --2- 172.21.15.99:0/81270377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810077c40 0x7f58100784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 --2- 172.21.15.99:0/81270377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f5810077700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 --2- 172.21.15.99:0/81270377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810076060 0x7f5810076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 -- 172.21.15.99:0/81270377 >> 172.21.15.99:0/81270377 conn(0x7f58100fee50 msgr2=0x7f5810101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 -- 172.21.15.99:0/81270377 shutdown_connections 2024-03-23T23:41:11.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.774+0000 7f5814e00640 1 -- 172.21.15.99:0/81270377 wait complete. 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 Processor -- start 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 -- start start 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810076060 0x7f581019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810077c40 0x7f58101a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58101139b0 con 0x7f5810077c40 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810113830 con 0x7f58100772a0 2024-03-23T23:41:11.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5814e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810113b30 con 0x7f5810076060 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5805400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5805400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40674/0 (socket says 172.21.15.99:40674) 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f5805400640 1 -- 172.21.15.99:0/4051842461 learned_addr learned my addr 172.21.15.99:0/4051842461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.775+0000 7f580e800640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810077c40 0x7f58101a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f580de00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810076060 0x7f581019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 -- 172.21.15.99:0/4051842461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810076060 msgr2=0x7f581019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810076060 0x7f581019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 -- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810077c40 msgr2=0x7f58101a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810077c40 0x7f58101a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 -- 172.21.15.99:0/4051842461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58101a5c10 con 0x7f58100772a0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5805400640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8043020 con 0x7f58100772a0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f80054e0 con 0x7f58100772a0 2024-03-23T23:41:11.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58101a5ea0 con 0x7f58100772a0 2024-03-23T23:41:11.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.776+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003ce0 con 0x7f58100772a0 2024-03-23T23:41:11.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.777+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58101a63b0 con 0x7f58100772a0 2024-03-23T23:41:11.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.778+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5810110250 con 0x7f58100772a0 2024-03-23T23:41:11.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.779+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f57f8046020 con 0x7f58100772a0 2024-03-23T23:41:11.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.780+0000 7f5807400640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 0x7f57f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:11.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.781+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57f80f0ab0 con 0x7f58100772a0 2024-03-23T23:41:11.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.781+0000 7f580de00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 0x7f57f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:11.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.781+0000 7f580de00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 0x7f57f40a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5800009b60 tx=0x7f5800005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:11.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:11.785+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f57f804d020 con 0x7f58100772a0 2024-03-23T23:41:12.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.023+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5810062340 con 0x7f58100772a0 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3932538955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:active 2024-03-23T23:41:12.613 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.614 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.614 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4051842461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.614 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:12 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3932538955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:active 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4051842461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.645 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:12 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.687 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3932538955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] up:active 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4051842461' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:12 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T23:41:12.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.675+0000 7f5807400640 1 -- 172.21.15.99:0/4051842461 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57f80bdc00 con 0x7f58100772a0 2024-03-23T23:41:12.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 msgr2=0x7f57f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:12.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 0x7f57f40a4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5800009b60 tx=0x7f5800005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:12.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 msgr2=0x7f581019e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:12.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f802c6c0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 shutdown_connections 2024-03-23T23:41:12.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f57f40a2880 0x7f57f40a4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5810077c40 0x7f58101a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58100772a0 0x7f581019e530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 --2- 172.21.15.99:0/4051842461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5810076060 0x7f581019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 >> 172.21.15.99:0/4051842461 conn(0x7f58100fee50 msgr2=0x7f5810100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:12.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 shutdown_connections 2024-03-23T23:41:12.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.679+0000 7f5814e00640 1 -- 172.21.15.99:0/4051842461 wait complete. 2024-03-23T23:41:12.699 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-23T23:41:12.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-23T23:41:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 -- 172.21.15.99:0/1138470061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88105490 msgr2=0x7fce8810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 --2- 172.21.15.99:0/1138470061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88105490 0x7fce8810bd20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce80009ca0 tx=0x7fce8002c0f0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 -- 172.21.15.99:0/1138470061 shutdown_connections 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 --2- 172.21.15.99:0/1138470061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88105490 0x7fce8810bd20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 --2- 172.21.15.99:0/1138470061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88104ad0 0x7fce88104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 --2- 172.21.15.99:0/1138470061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 -- 172.21.15.99:0/1138470061 >> 172.21.15.99:0/1138470061 conn(0x7fce880fee50 msgr2=0x7fce88101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 -- 172.21.15.99:0/1138470061 shutdown_connections 2024-03-23T23:41:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.908+0000 7fce8f000640 1 -- 172.21.15.99:0/1138470061 wait complete. 2024-03-23T23:41:12.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.909+0000 7fce8f000640 1 Processor -- start 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.909+0000 7fce8f000640 1 -- start start 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 0x7fce8819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88105490 0x7fce881a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810f560 con 0x7fce881038e0 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810f3e0 con 0x7fce88104ad0 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8f000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8810f6e0 con 0x7fce88105490 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce87e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 0x7fce8819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:12.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce87e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56094/0 (socket says 172.21.15.99:56094) 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce87e00640 1 -- 172.21.15.99:0/4165357189 learned_addr learned my addr 172.21.15.99:0/4165357189 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce8d000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88105490 0x7fce881a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 -- 172.21.15.99:0/4165357189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88105490 msgr2=0x7fce881a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88105490 0x7fce881a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 -- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 msgr2=0x7fce88199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.910+0000 7fce7f400640 1 -- 172.21.15.99:0/4165357189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce881a17f0 con 0x7fce88104ad0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce7f400640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 0x7fce8819a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e9f0 tx=0x7fce7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7000ccc0 con 0x7fce88104ad0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce70005020 con 0x7fce88104ad0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce881a1ae0 con 0x7fce88104ad0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700049e0 con 0x7fce88104ad0 2024-03-23T23:41:12.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.911+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce881a1fe0 con 0x7fce88104ad0 2024-03-23T23:41:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.912+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8810ac80 con 0x7fce88104ad0 2024-03-23T23:41:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.914+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fce70005560 con 0x7fce88104ad0 2024-03-23T23:41:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.915+0000 7fce85600640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 0x7fce6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.915+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fce700ce960 con 0x7fce88104ad0 2024-03-23T23:41:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.915+0000 7fce87e00640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 0x7fce6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.916+0000 7fce87e00640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 0x7fce6c0a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce780058c0 tx=0x7fce78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:12.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:12.919+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce700051d0 con 0x7fce88104ad0 2024-03-23T23:41:13.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.159+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fce8802d070 con 0x7fce88104ad0 2024-03-23T23:41:13.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.684+0000 7fce85600640 1 -- 172.21.15.99:0/4165357189 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fce7009bab0 con 0x7fce88104ad0 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 msgr2=0x7fce6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 0x7fce6c0a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce780058c0 tx=0x7fce78005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 msgr2=0x7fce8819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 0x7fce8819a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e9f0 tx=0x7fce7000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 shutdown_connections 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fce6c0a2800 0x7fce6c0a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce88105490 0x7fce881a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce88104ad0 0x7fce8819a110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 --2- 172.21.15.99:0/4165357189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce881038e0 0x7fce88199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 >> 172.21.15.99:0/4165357189 conn(0x7fce880fee50 msgr2=0x7fce88107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:13.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 shutdown_connections 2024-03-23T23:41:13.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.688+0000 7fce8f000640 1 -- 172.21.15.99:0/4165357189 wait complete. 2024-03-23T23:41:13.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 -- 172.21.15.99:0/187205083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5900772a0 msgr2=0x7fd590077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 --2- 172.21.15.99:0/187205083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5900772a0 0x7fd590077700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009900 tx=0x7fd57c02bc40 comp rx=0 tx=0).stop 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 -- 172.21.15.99:0/187205083 shutdown_connections 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 --2- 172.21.15.99:0/187205083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590077c40 0x7fd5900784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 --2- 172.21.15.99:0/187205083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5900772a0 0x7fd590077700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 --2- 172.21.15.99:0/187205083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd590076060 0x7fd590076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 -- 172.21.15.99:0/187205083 >> 172.21.15.99:0/187205083 conn(0x7fd5900fee50 msgr2=0x7fd590101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 -- 172.21.15.99:0/187205083 shutdown_connections 2024-03-23T23:41:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.903+0000 7fd595c00640 1 -- 172.21.15.99:0/187205083 wait complete. 2024-03-23T23:41:13.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.904+0000 7fd595c00640 1 Processor -- start 2024-03-23T23:41:13.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 -- start start 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590076060 0x7fd59019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 0x7fd59019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901139b0 con 0x7fd5900772a0 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590113830 con 0x7fd590077c40 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd595c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590113b30 con 0x7fd590076060 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 0x7fd59019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 0x7fd59019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56126/0 (socket says 172.21.15.99:56126) 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58e000640 1 -- 172.21.15.99:0/505924949 learned_addr learned my addr 172.21.15.99:0/505924949 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40716/0 (socket says 172.21.15.99:40716) 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58ea00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590076060 0x7fd59019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 -- 172.21.15.99:0/505924949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590076060 msgr2=0x7fd59019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:13.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590076060 0x7fd59019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 -- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 msgr2=0x7fd59019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.905+0000 7fd58f400640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 0x7fd59019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd58f400640 1 -- 172.21.15.99:0/505924949 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901a5c30 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd58f400640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d680 tx=0x7fd58800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588017070 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd588009820 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901a5f20 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588004e00 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.906+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901a63e0 con 0x7fd590077c40 2024-03-23T23:41:13.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.907+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590110250 con 0x7fd590077c40 2024-03-23T23:41:13.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.909+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd5880099d0 con 0x7fd590077c40 2024-03-23T23:41:13.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.910+0000 7fd587400640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:13.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.910+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5880ce210 con 0x7fd590077c40 2024-03-23T23:41:13.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.911+0000 7fd58ea00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:13.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.911+0000 7fd58ea00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 0x7fd5740a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd580005fd0 tx=0x7fd580009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:13.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:13.915+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd58809b360 con 0x7fd590077c40 2024-03-23T23:41:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:13 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T23:41:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:13 smithi099 ceph-mon[31715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:13 smithi099 ceph-mon[31715]: pgmap v318: 97 pgs: 55 active+clean, 24 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-23T23:41:14.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:13 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4165357189' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:13 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:13 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T23:41:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:13 smithi149 ceph-mon[36083]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:13 smithi149 ceph-mon[36083]: pgmap v318: 97 pgs: 55 active+clean, 24 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-23T23:41:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:13 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4165357189' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:13 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.152+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd590062340 con 0x7fd590077c40 2024-03-23T23:41:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:13 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T23:41:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:13 smithi174 ceph-mon[37002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:13 smithi174 ceph-mon[37002]: pgmap v318: 97 pgs: 55 active+clean, 24 creating+peering, 18 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-23T23:41:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:13 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4165357189' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:13 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-23T23:41:14.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.699+0000 7fd587400640 1 -- 172.21.15.99:0/505924949 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd58809ac60 con 0x7fd590077c40 2024-03-23T23:41:14.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 msgr2=0x7fd5740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:14.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 0x7fd5740a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd580005fd0 tx=0x7fd580009870 comp rx=0 tx=0).stop 2024-03-23T23:41:14.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 msgr2=0x7fd5901a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:14.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd58800d680 tx=0x7fd58800db40 comp rx=0 tx=0).stop 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 shutdown_connections 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd590077c40 0x7fd5901a55c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5900772a0 0x7fd59019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 --2- 172.21.15.99:0/505924949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd590076060 0x7fd59019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 >> 172.21.15.99:0/505924949 conn(0x7fd5900fee50 msgr2=0x7fd590100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 shutdown_connections 2024-03-23T23:41:14.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.704+0000 7fd595c00640 1 -- 172.21.15.99:0/505924949 wait complete. 2024-03-23T23:41:14.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.910+0000 7f5ad3e00640 1 -- 172.21.15.99:0/1015361195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad41015a0 msgr2=0x7f5ad41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.910+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/1015361195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad41015a0 0x7f5ad41019c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 -- 172.21.15.99:0/1015361195 shutdown_connections 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/1015361195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 0x7f5ad41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/1015361195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad4102790 0x7f5ad4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/1015361195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad41015a0 0x7f5ad41019c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 -- 172.21.15.99:0/1015361195 >> 172.21.15.99:0/1015361195 conn(0x7f5ad40fcb10 msgr2=0x7f5ad40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:14.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 -- 172.21.15.99:0/1015361195 shutdown_connections 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.911+0000 7f5ad3e00640 1 -- 172.21.15.99:0/1015361195 wait complete. 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 Processor -- start 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 -- start start 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad41015a0 0x7f5ad4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 0x7f5ad419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad410d260 con 0x7f5ad4103150 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad410d0e0 con 0x7f5ad4102790 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad410d3e0 con 0x7f5ad41015a0 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40732/0 (socket says 172.21.15.99:40732) 2024-03-23T23:41:14.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad2000640 1 -- 172.21.15.99:0/197270128 learned_addr learned my addr 172.21.15.99:0/197270128 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad3400640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 0x7f5ad419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.912+0000 7f5ad2a00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad41015a0 0x7f5ad4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 -- 172.21.15.99:0/197270128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad41015a0 msgr2=0x7f5ad4197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad41015a0 0x7f5ad4197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 -- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 msgr2=0x7f5ad419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:14.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 0x7f5ad419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:14.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 -- 172.21.15.99:0/197270128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad419f530 con 0x7f5ad4102790 2024-03-23T23:41:14.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad2000640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d770 tx=0x7f5ac000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017070 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ac0005500 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad419f820 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0004e00 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.913+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad419fd70 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.914+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad4108930 con 0x7f5ad4102790 2024-03-23T23:41:14.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.917+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5ac0004fa0 con 0x7f5ad4102790 2024-03-23T23:41:14.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.918+0000 7f5acb400640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 0x7f5ab80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:14.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.918+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ac00ce2e0 con 0x7f5ad4102790 2024-03-23T23:41:14.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.918+0000 7f5ad2a00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 0x7f5ab80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:14.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.919+0000 7f5ad2a00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 0x7f5ab80a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ac402c170 tx=0x7f5ac402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:14.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:14.922+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ac009ade0 con 0x7f5ad4102790 2024-03-23T23:41:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T23:41:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T23:41:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T23:41:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:15.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-23T23:41:15.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-23T23:41:15.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/505924949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:14 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.144 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:41:14.764+0000 7f1ba5dcb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T23:41:15.144 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:41:14.764+0000 7f1ba5dcb700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/505924949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:14 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T23:41:15.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-23T23:41:15.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T23:41:15.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T23:41:15.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-23T23:41:15.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-23T23:41:15.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/505924949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:14 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T23:41:15.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.173+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f5ad402d070 con 0x7f5ad4102790 2024-03-23T23:41:15.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.713+0000 7f5acb400640 1 -- 172.21.15.99:0/197270128 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5ac009ee80 con 0x7f5ad4102790 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.716+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 msgr2=0x7f5ab80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 0x7f5ab80a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ac402c170 tx=0x7f5ac402df90 comp rx=0 tx=0).stop 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 msgr2=0x7f5ad4197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000d770 tx=0x7f5ac000dc30 comp rx=0 tx=0).stop 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 shutdown_connections 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5ab80a2880 0x7f5ab80a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ad4103150 0x7f5ad419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ad4102790 0x7f5ad4197e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 --2- 172.21.15.99:0/197270128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ad41015a0 0x7f5ad4197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 >> 172.21.15.99:0/197270128 conn(0x7f5ad40fcb10 msgr2=0x7f5ad4105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 shutdown_connections 2024-03-23T23:41:15.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.717+0000 7f5ad3e00640 1 -- 172.21.15.99:0/197270128 wait complete. 2024-03-23T23:41:15.741 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-23T23:41:15.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-23T23:41:15.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 -- 172.21.15.99:0/311939305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8105470 msgr2=0x7fd0e810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:15.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 --2- 172.21.15.99:0/311939305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8105470 0x7fd0e810bd00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009be0 tx=0x7fd0e002bef0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 -- 172.21.15.99:0/311939305 shutdown_connections 2024-03-23T23:41:15.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 --2- 172.21.15.99:0/311939305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e8105470 0x7fd0e810bd00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 --2- 172.21.15.99:0/311939305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 0x7fd0e8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 --2- 172.21.15.99:0/311939305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e81038c0 0x7fd0e8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.931+0000 7fd0ef200640 1 -- 172.21.15.99:0/311939305 >> 172.21.15.99:0/311939305 conn(0x7fd0e80fee10 msgr2=0x7fd0e8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:15.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.932+0000 7fd0ef200640 1 -- 172.21.15.99:0/311939305 shutdown_connections 2024-03-23T23:41:15.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.932+0000 7fd0ef200640 1 -- 172.21.15.99:0/311939305 wait complete. 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 Processor -- start 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 -- start start 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 0x7fd0e819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 0x7fd0e81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e810f570 con 0x7fd0e81038c0 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e810f3f0 con 0x7fd0e8105470 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.933+0000 7fd0ef200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e810f6f0 con 0x7fd0e8104ab0 2024-03-23T23:41:15.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0e7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 0x7fd0e81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0e7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56148/0 (socket says 172.21.15.99:56148) 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0e7e00640 1 -- 172.21.15.99:0/2800779135 learned_addr learned my addr 172.21.15.99:0/2800779135 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0e7400640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 0x7fd0e819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 -- 172.21.15.99:0/2800779135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 msgr2=0x7fd0e819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 0x7fd0e819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 -- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 msgr2=0x7fd0e8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 -- 172.21.15.99:0/2800779135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e81a1740 con 0x7fd0e8105470 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0e7e00640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.934+0000 7fd0ed200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 0x7fd0e81a10d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0e002c430 tx=0x7fd0e00337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.935+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0043020 con 0x7fd0e8105470 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.935+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0e002ca60 con 0x7fd0e8105470 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.935+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e81a19d0 con 0x7fd0e8105470 2024-03-23T23:41:15.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.935+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0034690 con 0x7fd0e8105470 2024-03-23T23:41:15.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.935+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81a1f20 con 0x7fd0e8105470 2024-03-23T23:41:15.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.936+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e810ab80 con 0x7fd0e8105470 2024-03-23T23:41:15.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.939+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd0e0046020 con 0x7fd0e8105470 2024-03-23T23:41:15.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.941+0000 7fd0e4c00640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 0x7fd0cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:15.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.941+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0e00f0ec0 con 0x7fd0e8105470 2024-03-23T23:41:15.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.942+0000 7fd0e7e00640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 0x7fd0cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:15.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.942+0000 7fd0e7e00640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 0x7fd0cc0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80096c0 tx=0x7fd0d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:15.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:15.944+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0e004d020 con 0x7fd0e8105470 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:active 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-03-23T23:41:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: pgmap v319: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-23T23:41:16.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-03-23T23:41:16.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/197270128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:15 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:active 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: pgmap v319: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/197270128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:15 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] up:active 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:creating} 9 up:standby 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: pgmap v319: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-23T23:41:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-03-23T23:41:16.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/197270128' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:15 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T23:41:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.205+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0e8061ae0 con 0x7fd0cc0a2800 2024-03-23T23:41:16.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:41:16.215+0000 7f6fc1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T23:41:16.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:41:16.215+0000 7f6fc1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T23:41:16.612 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:41:16.215+0000 7f6fc1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T23:41:16.612 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:41:16.215+0000 7f6fc1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T23:41:16.612 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:41:16.215+0000 7f6fc1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T23:41:16.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.752+0000 7fd0e4c00640 1 -- 172.21.15.99:0/2800779135 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd0e8061ae0 con 0x7fd0cc0a2800 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 msgr2=0x7fd0cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 0x7fd0cc0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80096c0 tx=0x7fd0d8009260 comp rx=0 tx=0).stop 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 msgr2=0x7fd0e81a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 0x7fd0e81a10d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0e002c430 tx=0x7fd0e00337b0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 shutdown_connections 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd0cc0a2800 0x7fd0cc0a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0e8105470 0x7fd0e81a10d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104ab0 0x7fd0e819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 --2- 172.21.15.99:0/2800779135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0e81038c0 0x7fd0e8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 >> 172.21.15.99:0/2800779135 conn(0x7fd0e80fee10 msgr2=0x7fd0e8107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:16.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 shutdown_connections 2024-03-23T23:41:16.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.756+0000 7fd0ef200640 1 -- 172.21.15.99:0/2800779135 wait complete. 2024-03-23T23:41:16.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-23T23:41:16.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 -- 172.21.15.99:0/2490704332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104a60 msgr2=0x7f59f8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:16.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 --2- 172.21.15.99:0/2490704332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104a60 0x7f59f8106e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009900 tx=0x7f59e402bc50 comp rx=0 tx=0).stop 2024-03-23T23:41:16.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 -- 172.21.15.99:0/2490704332 shutdown_connections 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 --2- 172.21.15.99:0/2490704332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 0x7f59f81097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 --2- 172.21.15.99:0/2490704332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8104a60 0x7f59f8106e50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 --2- 172.21.15.99:0/2490704332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8102130 0x7f59f8104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 -- 172.21.15.99:0/2490704332 >> 172.21.15.99:0/2490704332 conn(0x7f59f80fbb40 msgr2=0x7f59f80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 -- 172.21.15.99:0/2490704332 shutdown_connections 2024-03-23T23:41:16.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.989+0000 7f59fe800640 1 -- 172.21.15.99:0/2490704332 wait complete. 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.990+0000 7f59fe800640 1 Processor -- start 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.990+0000 7f59fe800640 1 -- start start 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8104a60 0x7f59f8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 0x7f59f819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f810d370 con 0x7f59f8104a60 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f810d1f0 con 0x7f59f8107390 2024-03-23T23:41:16.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59fe800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f810d4f0 con 0x7f59f8102130 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50466/0 (socket says 172.21.15.99:50466) 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 -- 172.21.15.99:0/4164712061 learned_addr learned my addr 172.21.15.99:0/4164712061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f6a00640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8104a60 0x7f59f8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7e00640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 0x7f59f819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 -- 172.21.15.99:0/4164712061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 msgr2=0x7f59f819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 0x7f59f819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 -- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8104a60 msgr2=0x7f59f8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8104a60 0x7f59f8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.991+0000 7f59f7400640 1 -- 172.21.15.99:0/4164712061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f819f630 con 0x7f59f8102130 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59f7400640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59e800e9f0 tx=0x7f59e800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800ccc0 con 0x7f59f8102130 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59e8010910 con 0x7f59f8102130 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f819f920 con 0x7f59f8102130 2024-03-23T23:41:16.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e80188a0 con 0x7f59f8102130 2024-03-23T23:41:16.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.992+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f819fe90 con 0x7f59f8102130 2024-03-23T23:41:16.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.993+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f80fd630 con 0x7f59f8102130 2024-03-23T23:41:16.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.995+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f59e8010ac0 con 0x7f59f8102130 2024-03-23T23:41:16.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.996+0000 7f59efe00640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 0x7f59dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:16.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.997+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59e80ced50 con 0x7f59f8102130 2024-03-23T23:41:16.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.997+0000 7f59f6a00640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 0x7f59dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:16.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:16.998+0000 7f59f6a00640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 0x7f59dc0a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59e402c190 tx=0x7f59e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:17.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.000+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59e809bea0 con 0x7f59f8102130 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:active 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-03-23T23:41:17.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:16 smithi099 ceph-mon[31715]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:active 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-03-23T23:41:17.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:16 smithi149 ceph-mon[36083]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] up:active 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 9 up:standby 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:creating} 8 up:standby 2024-03-23T23:41:17.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:16 smithi174 ceph-mon[37002]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-03-23T23:41:17.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.298+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f8061ae0 con 0x7f59dc0a2880 2024-03-23T23:41:17.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.327+0000 7f59efe00640 1 -- 172.21.15.99:0/4164712061 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59f8061ae0 con 0x7f59dc0a2880 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 msgr2=0x7f59dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 0x7f59dc0a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59e402c190 tx=0x7f59e4000bc0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 msgr2=0x7f59f8197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59e800e9f0 tx=0x7f59e800edf0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 shutdown_connections 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f59dc0a2880 0x7f59dc0a4d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59f8107390 0x7f59f819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8104a60 0x7f59f8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 --2- 172.21.15.99:0/4164712061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f8102130 0x7f59f8197a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 >> 172.21.15.99:0/4164712061 conn(0x7f59f80fbb40 msgr2=0x7f59f8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:17.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 shutdown_connections 2024-03-23T23:41:17.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.330+0000 7f59fe800640 1 -- 172.21.15.99:0/4164712061 wait complete. 2024-03-23T23:41:17.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-23T23:41:17.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 -- 172.21.15.99:0/3761366505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 msgr2=0x7fd5380744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 --2- 172.21.15.99:0/3761366505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd5380744e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd520009900 tx=0x7fd52002bcd0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 -- 172.21.15.99:0/3761366505 shutdown_connections 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 --2- 172.21.15.99:0/3761366505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 0x7fd53810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 --2- 172.21.15.99:0/3761366505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd5380744e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 --2- 172.21.15.99:0/3761366505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 0x7fd538073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 -- 172.21.15.99:0/3761366505 >> 172.21.15.99:0/3761366505 conn(0x7fd5380fcb10 msgr2=0x7fd5380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 -- 172.21.15.99:0/3761366505 shutdown_connections 2024-03-23T23:41:17.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.542+0000 7fd53d200640 1 -- 172.21.15.99:0/3761366505 wait complete. 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.543+0000 7fd53d200640 1 Processor -- start 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 -- start start 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 0x7fd538199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 0x7fd5381a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53810f420 con 0x7fd538109450 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53810f2a0 con 0x7fd538074080 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd53d200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53810f5a0 con 0x7fd538075660 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd537400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd537400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40810/0 (socket says 172.21.15.99:40810) 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd537400640 1 -- 172.21.15.99:0/3371659339 learned_addr learned my addr 172.21.15.99:0/3371659339 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd536a00640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 0x7fd538199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:17.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.544+0000 7fd537e00640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 0x7fd5381a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 -- 172.21.15.99:0/3371659339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 msgr2=0x7fd538199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 0x7fd538199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 -- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 msgr2=0x7fd5381a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 0x7fd5381a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 -- 172.21.15.99:0/3371659339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5381a1670 con 0x7fd538074080 2024-03-23T23:41:17.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd537400640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd52800e9f0 tx=0x7fd52800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52800cd80 con 0x7fd538074080 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd528005500 con 0x7fd538074080 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5381a1960 con 0x7fd538074080 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.545+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528010690 con 0x7fd538074080 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.546+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5381a1e80 con 0x7fd538074080 2024-03-23T23:41:17.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.547+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd538108210 con 0x7fd538074080 2024-03-23T23:41:17.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.550+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd5280046c0 con 0x7fd538074080 2024-03-23T23:41:17.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.552+0000 7fd52fe00640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 0x7fd51c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:17.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.552+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5280cedc0 con 0x7fd538074080 2024-03-23T23:41:17.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.552+0000 7fd536a00640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 0x7fd51c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:17.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.553+0000 7fd536a00640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 0x7fd51c0a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd52002c850 tx=0x7fd520000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:17.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.555+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd52809bf10 con 0x7fd538074080 2024-03-23T23:41:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.747+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd528097e80 con 0x7fd538074080 2024-03-23T23:41:17.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.885+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd538061ae0 con 0x7fd51c0a2880 2024-03-23T23:41:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.893+0000 7fd52fe00640 1 -- 172.21.15.99:0/3371659339 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fd538061ae0 con 0x7fd51c0a2880 2024-03-23T23:41:17.893 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_0/4d534e87-fb6e-43c7-9fdb-b22482ce5b36 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 msgr2=0x7fd51c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 0x7fd51c0a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd52002c850 tx=0x7fd520000bc0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 msgr2=0x7fd538199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd52800e9f0 tx=0x7fd52800edf0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 shutdown_connections 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd51c0a2880 0x7fd51c0a4d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd538109450 0x7fd5381a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd538075660 0x7fd538199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 --2- 172.21.15.99:0/3371659339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd538074080 0x7fd538199aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 >> 172.21.15.99:0/3371659339 conn(0x7fd5380fcb10 msgr2=0x7fd5381053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 shutdown_connections 2024-03-23T23:41:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:17.896+0000 7fd53d200640 1 -- 172.21.15.99:0/3371659339 wait complete. 2024-03-23T23:41:17.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:active 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-23T23:41:17.979 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:active 2024-03-23T23:41:17.980 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:17 smithi099 ceph-mon[31715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 -- 172.21.15.99:0/2355121456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54104a60 msgr2=0x7f4d54106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 --2- 172.21.15.99:0/2355121456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54104a60 0x7f4d54106e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009860 tx=0x7f4d3c02bb70 comp rx=0 tx=0).stop 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 -- 172.21.15.99:0/2355121456 shutdown_connections 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 --2- 172.21.15.99:0/2355121456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 0x7f4d541097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 --2- 172.21.15.99:0/2355121456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54104a60 0x7f4d54106e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 --2- 172.21.15.99:0/2355121456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54102130 0x7f4d54104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 -- 172.21.15.99:0/2355121456 >> 172.21.15.99:0/2355121456 conn(0x7f4d540fbb40 msgr2=0x7f4d540fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:18.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 -- 172.21.15.99:0/2355121456 shutdown_connections 2024-03-23T23:41:18.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.110+0000 7f4d59000640 1 -- 172.21.15.99:0/2355121456 wait complete. 2024-03-23T23:41:18.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.111+0000 7f4d59000640 1 Processor -- start 2024-03-23T23:41:18.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.111+0000 7f4d59000640 1 -- start start 2024-03-23T23:41:18.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54104a60 0x7f4d541000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 0x7f4d54100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5410d330 con 0x7f4d54104a60 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5410d1b0 con 0x7f4d54107390 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d59000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5410d4b0 con 0x7f4d54102130 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d4be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54104a60 0x7f4d541000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50494/0 (socket says 172.21.15.99:50494) 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 -- 172.21.15.99:0/2748947680 learned_addr learned my addr 172.21.15.99:0/2748947680 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d52800640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 0x7f4d54100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 -- 172.21.15.99:0/2748947680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 msgr2=0x7f4d54100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 0x7f4d54100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 -- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54104a60 msgr2=0x7f4d541000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54104a60 0x7f4d541000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.112+0000 7f4d51e00640 1 -- 172.21.15.99:0/2748947680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54100eb0 con 0x7f4d54102130 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d51e00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e9c0 tx=0x7f4d4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400ccc0 con 0x7f4d54102130 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d44005500 con 0x7f4d54102130 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d541a1b50 con 0x7f4d54102130 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44010430 con 0x7f4d54102130 2024-03-23T23:41:18.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.113+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d541a2070 con 0x7f4d54102130 2024-03-23T23:41:18.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.115+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540688e0 con 0x7f4d54102130 2024-03-23T23:41:18.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.116+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4d440046c0 con 0x7f4d54102130 2024-03-23T23:41:18.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.117+0000 7f4d4aa00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 0x7f4d380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.117+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4d440cee80 con 0x7f4d54102130 2024-03-23T23:41:18.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.117+0000 7f4d4be00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 0x7f4d380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.118+0000 7f4d4be00640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 0x7f4d380a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0349e0 tx=0x7f4d3c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:18.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.121+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d4409be70 con 0x7f4d54102130 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:active 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-23T23:41:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:active 2024-03-23T23:41:18.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:17 smithi149 ceph-mon[36083]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:41:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 9 op/s 2024-03-23T23:41:18.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] up:active 2024-03-23T23:41:18.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-23T23:41:18.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T23:41:18.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:41:18.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=a=up:active,3=i=up:active} 8 up:standby 2024-03-23T23:41:18.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-23T23:41:18.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-23T23:41:18.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] up:active 2024-03-23T23:41:18.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:17 smithi174 ceph-mon[37002]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:41:18.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.366+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d54061c30 con 0x7f4d380a2890 2024-03-23T23:41:18.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.393+0000 7f4d4aa00640 1 -- 172.21.15.99:0/2748947680 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d54061c30 con 0x7f4d380a2890 2024-03-23T23:41:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 msgr2=0x7f4d380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 0x7f4d380a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0349e0 tx=0x7f4d3c034ab0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 msgr2=0x7f4d54101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e9c0 tx=0x7f4d4400ee80 comp rx=0 tx=0).stop 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 shutdown_connections 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4d380a2890 0x7f4d380a4d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54107390 0x7f4d54100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d54104a60 0x7f4d541000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 --2- 172.21.15.99:0/2748947680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102130 0x7f4d54101aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 >> 172.21.15.99:0/2748947680 conn(0x7f4d540fbb40 msgr2=0x7f4d54102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 shutdown_connections 2024-03-23T23:41:18.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.396+0000 7f4d59000640 1 -- 172.21.15.99:0/2748947680 wait complete. 2024-03-23T23:41:18.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.623+0000 7fcec3e00640 1 -- 172.21.15.99:0/2193563114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4109430 msgr2=0x7fcec410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.623+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2193563114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4109430 0x7fcec410b8d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009ca0 tx=0x7fcebc02c180 comp rx=0 tx=0).stop 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 -- 172.21.15.99:0/2193563114 shutdown_connections 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2193563114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4109430 0x7fcec410b8d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2193563114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 0x7fcec4074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2193563114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4075680 0x7fcec4073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 -- 172.21.15.99:0/2193563114 >> 172.21.15.99:0/2193563114 conn(0x7fcec40fcb10 msgr2=0x7fcec40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:18.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 -- 172.21.15.99:0/2193563114 shutdown_connections 2024-03-23T23:41:18.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.624+0000 7fcec3e00640 1 -- 172.21.15.99:0/2193563114 wait complete. 2024-03-23T23:41:18.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 Processor -- start 2024-03-23T23:41:18.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 -- start start 2024-03-23T23:41:18.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 0x7fcec4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4109430 0x7fcec41a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410f400 con 0x7fcec40740a0 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410f280 con 0x7fcec4075680 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410f580 con 0x7fcec4109430 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fceba000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fcec2a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 0x7fcec4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.625+0000 7fceba000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40838/0 (socket says 172.21.15.99:40838) 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 -- 172.21.15.99:0/2523358937 learned_addr learned my addr 172.21.15.99:0/2523358937 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fcec3400640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4109430 0x7fcec41a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 -- 172.21.15.99:0/2523358937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4109430 msgr2=0x7fcec41a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4109430 0x7fcec41a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 -- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 msgr2=0x7fcec4199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 0x7fcec4199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 -- 172.21.15.99:0/2523358937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec41a1630 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fceba000640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e9f0 tx=0x7fceac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00cd80 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fceac010910 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac018990 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.626+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec41a1920 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.627+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41a1e70 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.628+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec41081f0 con 0x7fcec4075680 2024-03-23T23:41:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.629+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fceac010ac0 con 0x7fcec4075680 2024-03-23T23:41:18.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.631+0000 7fcebbe00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 0x7fcea80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:18.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.631+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fceac0ceff0 con 0x7fcec4075680 2024-03-23T23:41:18.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.631+0000 7fcec2a00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 0x7fcea80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:18.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.631+0000 7fcec2a00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 0x7fcea80a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fceb40095f0 tx=0x7fceb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:18.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.637+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fceac09b990 con 0x7fcec4075680 2024-03-23T23:41:18.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.884+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcec4061ae0 con 0x7fcea80a2890 2024-03-23T23:41:18.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.892+0000 7fcebbe00640 1 -- 172.21.15.99:0/2523358937 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fcec4061ae0 con 0x7fcea80a2890 2024-03-23T23:41:18.892 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_1/b5b8b0ac-01ee-424f-ab5e-18f6d295a675 2024-03-23T23:41:18.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 msgr2=0x7fcea80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 0x7fcea80a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fceb40095f0 tx=0x7fceb4005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:18.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 msgr2=0x7fcec4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:18.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fceac00e9f0 tx=0x7fceac00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 shutdown_connections 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcea80a2890 0x7fcea80a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4109430 0x7fcec41a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcec4075680 0x7fcec4199f50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 --2- 172.21.15.99:0/2523358937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec40740a0 0x7fcec4199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 >> 172.21.15.99:0/2523358937 conn(0x7fcec40fcb10 msgr2=0x7fcec41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 shutdown_connections 2024-03-23T23:41:18.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:18.895+0000 7fcec3e00640 1 -- 172.21.15.99:0/2523358937 wait complete. 2024-03-23T23:41:18.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:41:19.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:18 smithi099 ceph-mon[31715]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:18 smithi099 ceph-mon[31715]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-23T23:41:19.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:18 smithi099 ceph-mon[31715]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:18 smithi149 ceph-mon[36083]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:18 smithi149 ceph-mon[36083]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-23T23:41:19.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:18 smithi149 ceph-mon[36083]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 -- 172.21.15.99:0/598635561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53381073c0 msgr2=0x7f5338109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 --2- 172.21.15.99:0/598635561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53381073c0 0x7f5338109810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5330009ca0 tx=0x7f533002c0f0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 -- 172.21.15.99:0/598635561 shutdown_connections 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 --2- 172.21.15.99:0/598635561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53381073c0 0x7f5338109810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 --2- 172.21.15.99:0/598635561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5338104a90 0x7f5338106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 --2- 172.21.15.99:0/598635561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102160 0x7f5338104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 -- 172.21.15.99:0/598635561 >> 172.21.15.99:0/598635561 conn(0x7f53380fbb50 msgr2=0x7f53380fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 -- 172.21.15.99:0/598635561 shutdown_connections 2024-03-23T23:41:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.154+0000 7f5340600640 1 -- 172.21.15.99:0/598635561 wait complete. 2024-03-23T23:41:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.155+0000 7f5340600640 1 Processor -- start 2024-03-23T23:41:19.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 -- start start 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338104a90 0x7f5338100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53381073c0 0x7f5338100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533810d380 con 0x7f53381073c0 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533810d200 con 0x7f5338102160 2024-03-23T23:41:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f5340600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533810d500 con 0x7f5338104a90 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f533dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f533dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42262/0 (socket says 172.21.15.99:42262) 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f533dc00640 1 -- 172.21.15.99:0/314309075 learned_addr learned my addr 172.21.15.99:0/314309075 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f533e600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53381073c0 0x7f5338100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.156+0000 7f533d200640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338104a90 0x7f5338100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 -- 172.21.15.99:0/314309075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338104a90 msgr2=0x7f5338100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338104a90 0x7f5338100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 -- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53381073c0 msgr2=0x7f5338100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53381073c0 0x7f5338100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 -- 172.21.15.99:0/314309075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338100ee0 con 0x7f5338102160 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f533dc00640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9f0 tx=0x7f532c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00cd80 con 0x7f5338102160 2024-03-23T23:41:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f532c005500 con 0x7f5338102160 2024-03-23T23:41:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c010690 con 0x7f5338102160 2024-03-23T23:41:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53381a1ae0 con 0x7f5338102160 2024-03-23T23:41:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.157+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53381a2030 con 0x7f5338102160 2024-03-23T23:41:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.159+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53380fd640 con 0x7f5338102160 2024-03-23T23:41:19.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.161+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f532c0046c0 con 0x7f5338102160 2024-03-23T23:41:19.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.162+0000 7f5336000640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 0x7f53200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.163+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f532c0ceee0 con 0x7f5338102160 2024-03-23T23:41:19.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.163+0000 7f533d200640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 0x7f53200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.163+0000 7f533d200640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 0x7f53200a4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5338101840 tx=0x7f5324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:19.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.171+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f532c09bed0 con 0x7f5338102160 2024-03-23T23:41:19.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:18 smithi174 ceph-mon[37002]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:18 smithi174 ceph-mon[37002]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-23T23:41:19.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:18 smithi174 ceph-mon[37002]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.414+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f533802d070 con 0x7f5338102160 2024-03-23T23:41:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.414+0000 7f5336000640 1 -- 172.21.15.99:0/314309075 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f532c09b7d0 con 0x7f5338102160 2024-03-23T23:41:19.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:19.415 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:41:19.415 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:41:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 msgr2=0x7f53200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 0x7f53200a4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5338101840 tx=0x7f5324005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 msgr2=0x7f5338101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f532c00e9f0 tx=0x7f532c00edf0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 shutdown_connections 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53200a2910 0x7f53200a4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53381073c0 0x7f5338100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338104a90 0x7f5338100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 --2- 172.21.15.99:0/314309075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5338102160 0x7f5338101ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 >> 172.21.15.99:0/314309075 conn(0x7f53380fbb50 msgr2=0x7f5338102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.417+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 shutdown_connections 2024-03-23T23:41:19.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.418+0000 7f5340600640 1 -- 172.21.15.99:0/314309075 wait complete. 2024-03-23T23:41:19.444 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 -- 172.21.15.99:0/3951539029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641048e0 msgr2=0x7fac64104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 --2- 172.21.15.99:0/3951539029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641048e0 0x7fac64104d60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac50009900 tx=0x7fac5002bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 -- 172.21.15.99:0/3951539029 shutdown_connections 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 --2- 172.21.15.99:0/3951539029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641052a0 0x7fac6410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 --2- 172.21.15.99:0/3951539029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641048e0 0x7fac64104d60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 --2- 172.21.15.99:0/3951539029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac64103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 -- 172.21.15.99:0/3951539029 >> 172.21.15.99:0/3951539029 conn(0x7fac640fec80 msgr2=0x7fac641010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:19.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 -- 172.21.15.99:0/3951539029 shutdown_connections 2024-03-23T23:41:19.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.680+0000 7fac69c00640 1 -- 172.21.15.99:0/3951539029 wait complete. 2024-03-23T23:41:19.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.681+0000 7fac69c00640 1 Processor -- start 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.681+0000 7fac69c00640 1 -- start start 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac641999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 0x7fac641a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6410f370 con 0x7fac641036f0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6410f1f0 con 0x7fac641052a0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6410f4f0 con 0x7fac641048e0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac641999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:36132/0 (socket says 172.21.15.99:36132) 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 -- 172.21.15.99:0/2743034507 learned_addr learned my addr 172.21.15.99:0/2743034507 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac69200640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 0x7fac641a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 -- 172.21.15.99:0/2743034507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 msgr2=0x7fac641a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 0x7fac641a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 -- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 msgr2=0x7fac641999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.682+0000 7fac63400640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac641999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac63400640 1 -- 172.21.15.99:0/2743034507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac641a1600 con 0x7fac641048e0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac63e00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac641999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac69200640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 0x7fac641a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac63400640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac50009230 tx=0x7fac5002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50041070 con 0x7fac641048e0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac50004450 con 0x7fac641048e0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac641a1890 con 0x7fac641048e0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50003e20 con 0x7fac641048e0 2024-03-23T23:41:19.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.683+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac641a1d80 con 0x7fac641048e0 2024-03-23T23:41:19.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.684+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6410aa90 con 0x7fac641048e0 2024-03-23T23:41:19.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.686+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fac50044070 con 0x7fac641048e0 2024-03-23T23:41:19.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.688+0000 7fac60c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 0x7fac480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:19.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.688+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fac500f05d0 con 0x7fac641048e0 2024-03-23T23:41:19.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.688+0000 7fac63e00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 0x7fac480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:19.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.688+0000 7fac63e00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 0x7fac480a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac64104740 tx=0x7fac54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:19.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.691+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac500bd080 con 0x7fac641048e0 2024-03-23T23:41:19.749 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:19 smithi099 ceph-mon[31715]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.750 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:19 smithi099 ceph-mon[31715]: pgmap v321: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-03-23T23:41:19.750 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:19 smithi099 ceph-mon[31715]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:19.750 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:19 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/314309075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:19.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.898+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac6402d2c0 con 0x7fac641048e0 2024-03-23T23:41:19.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.900+0000 7fac60c00640 1 -- 172.21.15.99:0/2743034507 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7fac500c1120 con 0x7fac641048e0 2024-03-23T23:41:19.900 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:19.900 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:41:11.233681+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.902+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 msgr2=0x7fac480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.902+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 0x7fac480a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac64104740 tx=0x7fac54005f90 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.902+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 msgr2=0x7fac64199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.902+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac50009230 tx=0x7fac5002c650 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 shutdown_connections 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fac480a2890 0x7fac480a4d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac641052a0 0x7fac641a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac641048e0 0x7fac64199f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 --2- 172.21.15.99:0/2743034507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac641036f0 0x7fac641999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 >> 172.21.15.99:0/2743034507 conn(0x7fac640fec80 msgr2=0x7fac64107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 shutdown_connections 2024-03-23T23:41:19.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:19.903+0000 7fac69c00640 1 -- 172.21.15.99:0/2743034507 wait complete. 2024-03-23T23:41:19.921 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T23:41:19.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T23:41:20.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 -- 172.21.15.99:0/3456920162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb90102160 msgr2=0x7fdb90104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 --2- 172.21.15.99:0/3456920162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb90102160 0x7fdb90104550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009900 tx=0x7fdb8002bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:20.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 -- 172.21.15.99:0/3456920162 shutdown_connections 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 --2- 172.21.15.99:0/3456920162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb901073c0 0x7fdb90109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 --2- 172.21.15.99:0/3456920162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90104a90 0x7fdb90106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 --2- 172.21.15.99:0/3456920162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb90102160 0x7fdb90104550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 -- 172.21.15.99:0/3456920162 >> 172.21.15.99:0/3456920162 conn(0x7fdb900fbb50 msgr2=0x7fdb900fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 -- 172.21.15.99:0/3456920162 shutdown_connections 2024-03-23T23:41:20.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.122+0000 7fdb96400640 1 -- 172.21.15.99:0/3456920162 wait complete. 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 Processor -- start 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 -- start start 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90102160 0x7fdb90199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb90104a90 0x7fdb9019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9010d3b0 con 0x7fdb90102160 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.123+0000 7fdb96400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9010d230 con 0x7fdb90104a90 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb96400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9010d530 con 0x7fdb901073c0 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90102160 0x7fdb90199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb90104a90 0x7fdb9019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:36150/0 (socket says 172.21.15.99:36150) 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 -- 172.21.15.99:0/3170623518 learned_addr learned my addr 172.21.15.99:0/3170623518 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 -- 172.21.15.99:0/3170623518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb90104a90 msgr2=0x7fdb9019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb90104a90 0x7fdb9019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 -- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90102160 msgr2=0x7fdb90199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90102160 0x7fdb90199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 -- 172.21.15.99:0/3170623518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb901a1850 con 0x7fdb901073c0 2024-03-23T23:41:20.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.124+0000 7fdb8fe00640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:20.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.125+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004b20 con 0x7fdb901073c0 2024-03-23T23:41:20.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.125+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb88004cc0 con 0x7fdb901073c0 2024-03-23T23:41:20.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.125+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88010760 con 0x7fdb901073c0 2024-03-23T23:41:20.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.125+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb901a1b40 con 0x7fdb901073c0 2024-03-23T23:41:20.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.125+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb901a2010 con 0x7fdb901073c0 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.126+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb90108f10 con 0x7fdb901073c0 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.129+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fdb88005020 con 0x7fdb901073c0 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.131+0000 7fdb87e00640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 0x7fdb740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.131+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb880cf040 con 0x7fdb901073c0 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.131+0000 7fdb8f400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 0x7fdb740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.132+0000 7fdb8f400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 0x7fdb740a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002c240 tx=0x7fdb8002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:20.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.139+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb8809c130 con 0x7fdb901073c0 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-mon[36083]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-mon[36083]: pgmap v321: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-mon[36083]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/314309075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:41:19.764+0000 7f1ba5dcb700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-23T23:41:20.144 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:41:19 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:41:19.764+0000 7f1ba5dcb700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-23T23:41:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:19 smithi174 ceph-mon[37002]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:19 smithi174 ceph-mon[37002]: pgmap v321: 97 pgs: 97 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.1 KiB/s wr, 23 op/s 2024-03-23T23:41:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:19 smithi174 ceph-mon[37002]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:41:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:19 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/314309075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:41:20.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.350+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb90061ae0 con 0x7fdb740a2890 2024-03-23T23:41:20.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.371+0000 7fdb87e00640 1 -- 172.21.15.99:0/3170623518 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258890 (secure 0 0 0) 0x7fdb90061ae0 con 0x7fdb740a2890 2024-03-23T23:41:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:20.377 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 msgr2=0x7fdb740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 0x7fdb740a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002c240 tx=0x7fdb8002df90 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 msgr2=0x7fdb901a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 shutdown_connections 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb740a2890 0x7fdb740a4d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb901073c0 0x7fdb901a11b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb90104a90 0x7fdb9019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 --2- 172.21.15.99:0/3170623518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb90102160 0x7fdb90199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 >> 172.21.15.99:0/3170623518 conn(0x7fdb900fbb50 msgr2=0x7fdb90101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 shutdown_connections 2024-03-23T23:41:20.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.377+0000 7fdb96400640 1 -- 172.21.15.99:0/3170623518 wait complete. 2024-03-23T23:41:20.397 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-03-23T23:41:18.726950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597302,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":353,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381088,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560736,"statfs":{"total":1151940427776,"available":1151550193664,"internally_reserved":0,"allocated":11403264,"data_stored":7051620,"data_compressed":107283,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6934,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":29,"num_read_kb":26,"num_write":172,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.491612"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599311+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:41:11.599311+0000","last_peered":"2024-03-23T23:41:11.599311+0000","last_clean":"2024-03-23T23:41:11.599311+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:41:11.599311+0000","last_undegraded":"2024-03-23T23:41:11.599311+0000","last_fullsized":"2024-03-23T23:41:11.599311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599264+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:41:11.599264+0000","last_peered":"2024-03-23T23:41:11.599264+0000","last_clean":"2024-03-23T23:41:11.599264+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:41:11.599264+0000","last_undegraded":"2024-03-23T23:41:11.599264+0000","last_fullsized":"2024-03-23T23:41:11.599264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599453+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:41:11.599453+0000","last_peered":"2024-03-23T23:41:11.599453+0000","last_clean":"2024-03-23T23:41:11.599453+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:41:11.599453+0000","last_undegraded":"2024-03-23T23:41:11.599453+0000","last_fullsized":"2024-03-23T23:41:11.599453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599310+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:41:11.599310+0000","last_peered":"2024-03-23T23:41:11.599310+0000","last_clean":"2024-03-23T23:41:11.599310+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:41:11.599310+0000","last_undegraded":"2024-03-23T23:41:11.599310+0000","last_fullsized":"2024-03-23T23:41:11.599310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599270+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:41:11.599270+0000","last_peered":"2024-03-23T23:41:11.599270+0000","last_clean":"2024-03-23T23:41:11.599270+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:41:11.599270+0000","last_undegraded":"2024-03-23T23:41:11.599270+0000","last_fullsized":"2024-03-23T23:41:11.599270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599565+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:41:11.599565+0000","last_peered":"2024-03-23T23:41:11.599565+0000","last_clean":"2024-03-23T23:41:11.599565+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:41:11.599565+0000","last_undegraded":"2024-03-23T23:41:11.599565+0000","last_fullsized":"2024-03-23T23:41:11.599565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861675+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:41:11.861675+0000","last_peered":"2024-03-23T23:41:11.861675+0000","last_clean":"2024-03-23T23:41:11.861675+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:41:11.861675+0000","last_undegraded":"2024-03-23T23:41:11.861675+0000","last_fullsized":"2024-03-23T23:41:11.861675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599074+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:41:11.599074+0000","last_peered":"2024-03-23T23:41:11.599074+0000","last_clean":"2024-03-23T23:41:11.599074+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:41:11.599074+0000","last_undegraded":"2024-03-23T23:41:11.599074+0000","last_fullsized":"2024-03-23T23:41:11.599074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240173+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:41:11.240173+0000","last_peered":"2024-03-23T23:41:11.240173+0000","last_clean":"2024-03-23T23:41:11.240173+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:41:11.240173+0000","last_undegraded":"2024-03-23T23:41:11.240173+0000","last_fullsized":"2024-03-23T23:41:11.240173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599876+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:41:11.599876+0000","last_peered":"2024-03-23T23:41:11.599876+0000","last_clean":"2024-03-23T23:41:11.599876+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:41:11.599876+0000","last_undegraded":"2024-03-23T23:41:11.599876+0000","last_fullsized":"2024-03-23T23:41:11.599876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.862021+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:41:11.862021+0000","last_peered":"2024-03-23T23:41:11.862021+0000","last_clean":"2024-03-23T23:41:11.862021+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:41:11.862021+0000","last_undegraded":"2024-03-23T23:41:11.862021+0000","last_fullsized":"2024-03-23T23:41:11.862021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599195+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:41:11.599195+0000","last_peered":"2024-03-23T23:41:11.599195+0000","last_clean":"2024-03-23T23:41:11.599195+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:41:11.599195+0000","last_undegraded":"2024-03-23T23:41:11.599195+0000","last_fullsized":"2024-03-23T23:41:11.599195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388827+0000","last_change":"2024-03-23T23:41:10.276300+0000","last_active":"2024-03-23T23:41:11.388827+0000","last_peered":"2024-03-23T23:41:11.388827+0000","last_clean":"2024-03-23T23:41:11.388827+0000","last_became_active":"2024-03-23T23:41:10.275762+0000","last_became_peered":"2024-03-23T23:41:10.275762+0000","last_unstale":"2024-03-23T23:41:11.388827+0000","last_undegraded":"2024-03-23T23:41:11.388827+0000","last_fullsized":"2024-03-23T23:41:11.388827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.705104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598731+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:41:11.598731+0000","last_peered":"2024-03-23T23:41:11.598731+0000","last_clean":"2024-03-23T23:41:11.598731+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:41:11.598731+0000","last_undegraded":"2024-03-23T23:41:11.598731+0000","last_fullsized":"2024-03-23T23:41:11.598731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598627+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:41:11.598627+0000","last_peered":"2024-03-23T23:41:11.598627+0000","last_clean":"2024-03-23T23:41:11.598627+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:41:11.598627+0000","last_undegraded":"2024-03-23T23:41:11.598627+0000","last_fullsized":"2024-03-23T23:41:11.598627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239607+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:41:11.239607+0000","last_peered":"2024-03-23T23:41:11.239607+0000","last_clean":"2024-03-23T23:41:11.239607+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:41:11.239607+0000","last_undegraded":"2024-03-23T23:41:11.239607+0000","last_fullsized":"2024-03-23T23:41:11.239607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599325+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:41:11.599325+0000","last_peered":"2024-03-23T23:41:11.599325+0000","last_clean":"2024-03-23T23:41:11.599325+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:41:11.599325+0000","last_undegraded":"2024-03-23T23:41:11.599325+0000","last_fullsized":"2024-03-23T23:41:11.599325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.241446+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:41:11.241446+0000","last_peered":"2024-03-23T23:41:11.241446+0000","last_clean":"2024-03-23T23:41:11.241446+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:41:11.241446+0000","last_undegraded":"2024-03-23T23:41:11.241446+0000","last_fullsized":"2024-03-23T23:41:11.241446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242186+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:41:11.242186+0000","last_peered":"2024-03-23T23:41:11.242186+0000","last_clean":"2024-03-23T23:41:11.242186+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:41:11.242186+0000","last_undegraded":"2024-03-23T23:41:11.242186+0000","last_fullsized":"2024-03-23T23:41:11.242186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.291026+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:41:17.291026+0000","last_peered":"2024-03-23T23:41:17.291026+0000","last_clean":"2024-03-23T23:41:17.291026+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:41:17.291026+0000","last_undegraded":"2024-03-23T23:41:17.291026+0000","last_fullsized":"2024-03-23T23:41:17.291026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599668+0000","last_change":"2024-03-23T23:41:10.274414+0000","last_active":"2024-03-23T23:41:11.599668+0000","last_peered":"2024-03-23T23:41:11.599668+0000","last_clean":"2024-03-23T23:41:11.599668+0000","last_became_active":"2024-03-23T23:41:10.272770+0000","last_became_peered":"2024-03-23T23:41:10.272770+0000","last_unstale":"2024-03-23T23:41:11.599668+0000","last_undegraded":"2024-03-23T23:41:11.599668+0000","last_fullsized":"2024-03-23T23:41:11.599668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:02:51.844280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600320+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:41:11.600320+0000","last_peered":"2024-03-23T23:41:11.600320+0000","last_clean":"2024-03-23T23:41:11.600320+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:41:11.600320+0000","last_undegraded":"2024-03-23T23:41:11.600320+0000","last_fullsized":"2024-03-23T23:41:11.600320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599260+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:41:11.599260+0000","last_peered":"2024-03-23T23:41:11.599260+0000","last_clean":"2024-03-23T23:41:11.599260+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:41:11.599260+0000","last_undegraded":"2024-03-23T23:41:11.599260+0000","last_fullsized":"2024-03-23T23:41:11.599260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.288093+0000","last_change":"2024-03-23T23:41:09.245775+0000","last_active":"2024-03-23T23:41:17.288093+0000","last_peered":"2024-03-23T23:41:17.288093+0000","last_clean":"2024-03-23T23:41:17.288093+0000","last_became_active":"2024-03-23T23:41:09.245406+0000","last_became_peered":"2024-03-23T23:41:09.245406+0000","last_unstale":"2024-03-23T23:41:17.288093+0000","last_undegraded":"2024-03-23T23:41:17.288093+0000","last_fullsized":"2024-03-23T23:41:17.288093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:31:40.079757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239669+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:41:11.239669+0000","last_peered":"2024-03-23T23:41:11.239669+0000","last_clean":"2024-03-23T23:41:11.239669+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:41:11.239669+0000","last_undegraded":"2024-03-23T23:41:11.239669+0000","last_fullsized":"2024-03-23T23:41:11.239669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.744693+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:41:16.744693+0000","last_peered":"2024-03-23T23:41:16.744693+0000","last_clean":"2024-03-23T23:41:16.744693+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:41:16.744693+0000","last_undegraded":"2024-03-23T23:41:16.744693+0000","last_fullsized":"2024-03-23T23:41:16.744693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599668+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:41:11.599668+0000","last_peered":"2024-03-23T23:41:11.599668+0000","last_clean":"2024-03-23T23:41:11.599668+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:41:11.599668+0000","last_undegraded":"2024-03-23T23:41:11.599668+0000","last_fullsized":"2024-03-23T23:41:11.599668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.289187+0000","last_change":"2024-03-23T23:41:09.251958+0000","last_active":"2024-03-23T23:41:17.289187+0000","last_peered":"2024-03-23T23:41:17.289187+0000","last_clean":"2024-03-23T23:41:17.289187+0000","last_became_active":"2024-03-23T23:41:09.251609+0000","last_became_peered":"2024-03-23T23:41:09.251609+0000","last_unstale":"2024-03-23T23:41:17.289187+0000","last_undegraded":"2024-03-23T23:41:17.289187+0000","last_fullsized":"2024-03-23T23:41:17.289187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:46:43.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599652+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:41:11.599652+0000","last_peered":"2024-03-23T23:41:11.599652+0000","last_clean":"2024-03-23T23:41:11.599652+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:41:11.599652+0000","last_undegraded":"2024-03-23T23:41:11.599652+0000","last_fullsized":"2024-03-23T23:41:11.599652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.722346+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:41:14.722346+0000","last_peered":"2024-03-23T23:41:14.722346+0000","last_clean":"2024-03-23T23:41:14.722346+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:41:14.722346+0000","last_undegraded":"2024-03-23T23:41:14.722346+0000","last_fullsized":"2024-03-23T23:41:14.722346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599711+0000","last_change":"2024-03-23T23:41:10.270493+0000","last_active":"2024-03-23T23:41:11.599711+0000","last_peered":"2024-03-23T23:41:11.599711+0000","last_clean":"2024-03-23T23:41:11.599711+0000","last_became_active":"2024-03-23T23:41:10.270141+0000","last_became_peered":"2024-03-23T23:41:10.270141+0000","last_unstale":"2024-03-23T23:41:11.599711+0000","last_undegraded":"2024-03-23T23:41:11.599711+0000","last_fullsized":"2024-03-23T23:41:11.599711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:55.277951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.729253+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:41:14.729253+0000","last_peered":"2024-03-23T23:41:14.729253+0000","last_clean":"2024-03-23T23:41:14.729253+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:41:14.729253+0000","last_undegraded":"2024-03-23T23:41:14.729253+0000","last_fullsized":"2024-03-23T23:41:14.729253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598896+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:41:11.598896+0000","last_peered":"2024-03-23T23:41:11.598896+0000","last_clean":"2024-03-23T23:41:11.598896+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:41:11.598896+0000","last_undegraded":"2024-03-23T23:41:11.598896+0000","last_fullsized":"2024-03-23T23:41:11.598896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.745257+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:41:16.745257+0000","last_peered":"2024-03-23T23:41:16.745257+0000","last_clean":"2024-03-23T23:41:16.745257+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:41:16.745257+0000","last_undegraded":"2024-03-23T23:41:16.745257+0000","last_fullsized":"2024-03-23T23:41:16.745257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240694+0000","last_change":"2024-03-23T23:41:10.271143+0000","last_active":"2024-03-23T23:41:11.240694+0000","last_peered":"2024-03-23T23:41:11.240694+0000","last_clean":"2024-03-23T23:41:11.240694+0000","last_became_active":"2024-03-23T23:41:10.270678+0000","last_became_peered":"2024-03-23T23:41:10.270678+0000","last_unstale":"2024-03-23T23:41:11.240694+0000","last_undegraded":"2024-03-23T23:41:11.240694+0000","last_fullsized":"2024-03-23T23:41:11.240694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:01:36.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.707911+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:41:13.707911+0000","last_peered":"2024-03-23T23:41:13.707911+0000","last_clean":"2024-03-23T23:41:13.707911+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:41:13.707911+0000","last_undegraded":"2024-03-23T23:41:13.707911+0000","last_fullsized":"2024-03-23T23:41:13.707911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240047+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:41:11.240047+0000","last_peered":"2024-03-23T23:41:11.240047+0000","last_clean":"2024-03-23T23:41:11.240047+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:41:11.240047+0000","last_undegraded":"2024-03-23T23:41:11.240047+0000","last_fullsized":"2024-03-23T23:41:11.240047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.707175+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:41:13.707175+0000","last_peered":"2024-03-23T23:41:13.707175+0000","last_clean":"2024-03-23T23:41:13.707175+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:41:13.707175+0000","last_undegraded":"2024-03-23T23:41:13.707175+0000","last_fullsized":"2024-03-23T23:41:13.707175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599983+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:41:11.599983+0000","last_peered":"2024-03-23T23:41:11.599983+0000","last_clean":"2024-03-23T23:41:11.599983+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:41:11.599983+0000","last_undegraded":"2024-03-23T23:41:11.599983+0000","last_fullsized":"2024-03-23T23:41:11.599983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.729693+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:41:15.729693+0000","last_peered":"2024-03-23T23:41:15.729693+0000","last_clean":"2024-03-23T23:41:15.729693+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:41:15.729693+0000","last_undegraded":"2024-03-23T23:41:15.729693+0000","last_fullsized":"2024-03-23T23:41:15.729693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239975+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:41:11.239975+0000","last_peered":"2024-03-23T23:41:11.239975+0000","last_clean":"2024-03-23T23:41:11.239975+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:41:11.239975+0000","last_undegraded":"2024-03-23T23:41:11.239975+0000","last_fullsized":"2024-03-23T23:41:11.239975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.707758+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:41:13.707758+0000","last_peered":"2024-03-23T23:41:13.707758+0000","last_clean":"2024-03-23T23:41:13.707758+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:41:13.707758+0000","last_undegraded":"2024-03-23T23:41:13.707758+0000","last_fullsized":"2024-03-23T23:41:13.707758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600107+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:41:11.600107+0000","last_peered":"2024-03-23T23:41:11.600107+0000","last_clean":"2024-03-23T23:41:11.600107+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:41:11.600107+0000","last_undegraded":"2024-03-23T23:41:11.600107+0000","last_fullsized":"2024-03-23T23:41:11.600107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.709501+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:41:13.709501+0000","last_peered":"2024-03-23T23:41:13.709501+0000","last_clean":"2024-03-23T23:41:13.709501+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:41:13.709501+0000","last_undegraded":"2024-03-23T23:41:13.709501+0000","last_fullsized":"2024-03-23T23:41:13.709501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388678+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:41:11.388678+0000","last_peered":"2024-03-23T23:41:11.388678+0000","last_clean":"2024-03-23T23:41:11.388678+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:41:11.388678+0000","last_undegraded":"2024-03-23T23:41:11.388678+0000","last_fullsized":"2024-03-23T23:41:11.388678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748360+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:41:16.748360+0000","last_peered":"2024-03-23T23:41:16.748360+0000","last_clean":"2024-03-23T23:41:16.748360+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:41:16.748360+0000","last_undegraded":"2024-03-23T23:41:16.748360+0000","last_fullsized":"2024-03-23T23:41:16.748360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861891+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:41:11.861891+0000","last_peered":"2024-03-23T23:41:11.861891+0000","last_clean":"2024-03-23T23:41:11.861891+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:41:11.861891+0000","last_undegraded":"2024-03-23T23:41:11.861891+0000","last_fullsized":"2024-03-23T23:41:11.861891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.291714+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:41:17.291714+0000","last_peered":"2024-03-23T23:41:17.291714+0000","last_clean":"2024-03-23T23:41:17.291714+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:41:17.291714+0000","last_undegraded":"2024-03-23T23:41:17.291714+0000","last_fullsized":"2024-03-23T23:41:17.291714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861626+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:41:11.861626+0000","last_peered":"2024-03-23T23:41:11.861626+0000","last_clean":"2024-03-23T23:41:11.861626+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:41:11.861626+0000","last_undegraded":"2024-03-23T23:41:11.861626+0000","last_fullsized":"2024-03-23T23:41:11.861626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.747007+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:41:16.747007+0000","last_peered":"2024-03-23T23:41:16.747007+0000","last_clean":"2024-03-23T23:41:16.747007+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:41:16.747007+0000","last_undegraded":"2024-03-23T23:41:16.747007+0000","last_fullsized":"2024-03-23T23:41:16.747007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599614+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:41:11.599614+0000","last_peered":"2024-03-23T23:41:11.599614+0000","last_clean":"2024-03-23T23:41:11.599614+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:41:11.599614+0000","last_undegraded":"2024-03-23T23:41:11.599614+0000","last_fullsized":"2024-03-23T23:41:11.599614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.717461+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:41:15.717461+0000","last_peered":"2024-03-23T23:41:15.717461+0000","last_clean":"2024-03-23T23:41:15.717461+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:41:15.717461+0000","last_undegraded":"2024-03-23T23:41:15.717461+0000","last_fullsized":"2024-03-23T23:41:15.717461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599183+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:41:11.599183+0000","last_peered":"2024-03-23T23:41:11.599183+0000","last_clean":"2024-03-23T23:41:11.599183+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:41:11.599183+0000","last_undegraded":"2024-03-23T23:41:11.599183+0000","last_fullsized":"2024-03-23T23:41:11.599183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240749+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:41:11.240749+0000","last_peered":"2024-03-23T23:41:11.240749+0000","last_clean":"2024-03-23T23:41:11.240749+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:41:11.240749+0000","last_undegraded":"2024-03-23T23:41:11.240749+0000","last_fullsized":"2024-03-23T23:41:11.240749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599750+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:41:11.599750+0000","last_peered":"2024-03-23T23:41:11.599750+0000","last_clean":"2024-03-23T23:41:11.599750+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:41:11.599750+0000","last_undegraded":"2024-03-23T23:41:11.599750+0000","last_fullsized":"2024-03-23T23:41:11.599750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861736+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:41:11.861736+0000","last_peered":"2024-03-23T23:41:11.861736+0000","last_clean":"2024-03-23T23:41:11.861736+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:41:11.861736+0000","last_undegraded":"2024-03-23T23:41:11.861736+0000","last_fullsized":"2024-03-23T23:41:11.861736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599590+0000","last_change":"2024-03-23T23:41:10.286442+0000","last_active":"2024-03-23T23:41:11.599590+0000","last_peered":"2024-03-23T23:41:11.599590+0000","last_clean":"2024-03-23T23:41:11.599590+0000","last_became_active":"2024-03-23T23:41:10.286085+0000","last_became_peered":"2024-03-23T23:41:10.286085+0000","last_unstale":"2024-03-23T23:41:11.599590+0000","last_undegraded":"2024-03-23T23:41:11.599590+0000","last_fullsized":"2024-03-23T23:41:11.599590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:58.730743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599539+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:41:11.599539+0000","last_peered":"2024-03-23T23:41:11.599539+0000","last_clean":"2024-03-23T23:41:11.599539+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:41:11.599539+0000","last_undegraded":"2024-03-23T23:41:11.599539+0000","last_fullsized":"2024-03-23T23:41:11.599539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.729834+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:41:14.729834+0000","last_peered":"2024-03-23T23:41:14.729834+0000","last_clean":"2024-03-23T23:41:14.729834+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:41:14.729834+0000","last_undegraded":"2024-03-23T23:41:14.729834+0000","last_fullsized":"2024-03-23T23:41:14.729834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599352+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:41:11.599352+0000","last_peered":"2024-03-23T23:41:11.599352+0000","last_clean":"2024-03-23T23:41:11.599352+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:41:11.599352+0000","last_undegraded":"2024-03-23T23:41:11.599352+0000","last_fullsized":"2024-03-23T23:41:11.599352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.288109+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:41:17.288109+0000","last_peered":"2024-03-23T23:41:17.288109+0000","last_clean":"2024-03-23T23:41:17.288109+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:41:17.288109+0000","last_undegraded":"2024-03-23T23:41:17.288109+0000","last_fullsized":"2024-03-23T23:41:17.288109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861624+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:41:11.861624+0000","last_peered":"2024-03-23T23:41:11.861624+0000","last_clean":"2024-03-23T23:41:11.861624+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:41:11.861624+0000","last_undegraded":"2024-03-23T23:41:11.861624+0000","last_fullsized":"2024-03-23T23:41:11.861624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.387944+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:41:11.387944+0000","last_peered":"2024-03-23T23:41:11.387944+0000","last_clean":"2024-03-23T23:41:11.387944+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:41:11.387944+0000","last_undegraded":"2024-03-23T23:41:11.387944+0000","last_fullsized":"2024-03-23T23:41:11.387944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.722402+0000","last_change":"2024-03-23T23:41:09.252221+0000","last_active":"2024-03-23T23:41:14.722402+0000","last_peered":"2024-03-23T23:41:14.722402+0000","last_clean":"2024-03-23T23:41:14.722402+0000","last_became_active":"2024-03-23T23:41:09.251930+0000","last_became_peered":"2024-03-23T23:41:09.251930+0000","last_unstale":"2024-03-23T23:41:14.722402+0000","last_undegraded":"2024-03-23T23:41:14.722402+0000","last_fullsized":"2024-03-23T23:41:14.722402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:31.262380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748873+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:41:16.748873+0000","last_peered":"2024-03-23T23:41:16.748873+0000","last_clean":"2024-03-23T23:41:16.748873+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:41:16.748873+0000","last_undegraded":"2024-03-23T23:41:16.748873+0000","last_fullsized":"2024-03-23T23:41:16.748873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388365+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:41:11.388365+0000","last_peered":"2024-03-23T23:41:11.388365+0000","last_clean":"2024-03-23T23:41:11.388365+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:41:11.388365+0000","last_undegraded":"2024-03-23T23:41:11.388365+0000","last_fullsized":"2024-03-23T23:41:11.388365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.735482+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:41:15.735482+0000","last_peered":"2024-03-23T23:41:15.735482+0000","last_clean":"2024-03-23T23:41:15.735482+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:41:15.735482+0000","last_undegraded":"2024-03-23T23:41:15.735482+0000","last_fullsized":"2024-03-23T23:41:15.735482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600480+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:41:11.600480+0000","last_peered":"2024-03-23T23:41:11.600480+0000","last_clean":"2024-03-23T23:41:11.600480+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:41:11.600480+0000","last_undegraded":"2024-03-23T23:41:11.600480+0000","last_fullsized":"2024-03-23T23:41:11.600480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.747226+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:41:16.747226+0000","last_peered":"2024-03-23T23:41:16.747226+0000","last_clean":"2024-03-23T23:41:16.747226+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:41:16.747226+0000","last_undegraded":"2024-03-23T23:41:16.747226+0000","last_fullsized":"2024-03-23T23:41:16.747226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599428+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:41:11.599428+0000","last_peered":"2024-03-23T23:41:11.599428+0000","last_clean":"2024-03-23T23:41:11.599428+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:41:11.599428+0000","last_undegraded":"2024-03-23T23:41:11.599428+0000","last_fullsized":"2024-03-23T23:41:11.599428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.706709+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:41:13.706709+0000","last_peered":"2024-03-23T23:41:13.706709+0000","last_clean":"2024-03-23T23:41:13.706709+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:41:13.706709+0000","last_undegraded":"2024-03-23T23:41:13.706709+0000","last_fullsized":"2024-03-23T23:41:13.706709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598870+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:41:11.598870+0000","last_peered":"2024-03-23T23:41:11.598870+0000","last_clean":"2024-03-23T23:41:11.598870+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:41:11.598870+0000","last_undegraded":"2024-03-23T23:41:11.598870+0000","last_fullsized":"2024-03-23T23:41:11.598870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.709145+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:41:13.709145+0000","last_peered":"2024-03-23T23:41:13.709145+0000","last_clean":"2024-03-23T23:41:13.709145+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:41:13.709145+0000","last_undegraded":"2024-03-23T23:41:13.709145+0000","last_fullsized":"2024-03-23T23:41:13.709145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598964+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:41:11.598964+0000","last_peered":"2024-03-23T23:41:11.598964+0000","last_clean":"2024-03-23T23:41:11.598964+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:41:11.598964+0000","last_undegraded":"2024-03-23T23:41:11.598964+0000","last_fullsized":"2024-03-23T23:41:11.598964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.241875+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:41:11.241875+0000","last_peered":"2024-03-23T23:41:11.241875+0000","last_clean":"2024-03-23T23:41:11.241875+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:41:11.241875+0000","last_undegraded":"2024-03-23T23:41:11.241875+0000","last_fullsized":"2024-03-23T23:41:11.241875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599932+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:41:11.599932+0000","last_peered":"2024-03-23T23:41:11.599932+0000","last_clean":"2024-03-23T23:41:11.599932+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:41:11.599932+0000","last_undegraded":"2024-03-23T23:41:11.599932+0000","last_fullsized":"2024-03-23T23:41:11.599932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.707796+0000","last_change":"2024-03-23T23:41:09.247532+0000","last_active":"2024-03-23T23:41:13.707796+0000","last_peered":"2024-03-23T23:41:13.707796+0000","last_clean":"2024-03-23T23:41:13.707796+0000","last_became_active":"2024-03-23T23:41:09.245141+0000","last_became_peered":"2024-03-23T23:41:09.245141+0000","last_unstale":"2024-03-23T23:41:13.707796+0000","last_undegraded":"2024-03-23T23:41:13.707796+0000","last_fullsized":"2024-03-23T23:41:13.707796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:05:42.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861961+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:41:11.861961+0000","last_peered":"2024-03-23T23:41:11.861961+0000","last_clean":"2024-03-23T23:41:11.861961+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:41:11.861961+0000","last_undegraded":"2024-03-23T23:41:11.861961+0000","last_fullsized":"2024-03-23T23:41:11.861961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242275+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:41:11.242275+0000","last_peered":"2024-03-23T23:41:11.242275+0000","last_clean":"2024-03-23T23:41:11.242275+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:41:11.242275+0000","last_undegraded":"2024-03-23T23:41:11.242275+0000","last_fullsized":"2024-03-23T23:41:11.242275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.862038+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:41:11.862038+0000","last_peered":"2024-03-23T23:41:11.862038+0000","last_clean":"2024-03-23T23:41:11.862038+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:41:11.862038+0000","last_undegraded":"2024-03-23T23:41:11.862038+0000","last_fullsized":"2024-03-23T23:41:11.862038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748382+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:41:16.748382+0000","last_peered":"2024-03-23T23:41:16.748382+0000","last_clean":"2024-03-23T23:41:16.748382+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:41:16.748382+0000","last_undegraded":"2024-03-23T23:41:16.748382+0000","last_fullsized":"2024-03-23T23:41:16.748382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861877+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:41:11.861877+0000","last_peered":"2024-03-23T23:41:11.861877+0000","last_clean":"2024-03-23T23:41:11.861877+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:41:11.861877+0000","last_undegraded":"2024-03-23T23:41:11.861877+0000","last_fullsized":"2024-03-23T23:41:11.861877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748076+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:41:16.748076+0000","last_peered":"2024-03-23T23:41:16.748076+0000","last_clean":"2024-03-23T23:41:16.748076+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:41:16.748076+0000","last_undegraded":"2024-03-23T23:41:16.748076+0000","last_fullsized":"2024-03-23T23:41:16.748076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861968+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:41:11.861968+0000","last_peered":"2024-03-23T23:41:11.861968+0000","last_clean":"2024-03-23T23:41:11.861968+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:41:11.861968+0000","last_undegraded":"2024-03-23T23:41:11.861968+0000","last_fullsized":"2024-03-23T23:41:11.861968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599238+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:41:11.599238+0000","last_peered":"2024-03-23T23:41:11.599238+0000","last_clean":"2024-03-23T23:41:11.599238+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:41:11.599238+0000","last_undegraded":"2024-03-23T23:41:11.599238+0000","last_fullsized":"2024-03-23T23:41:11.599238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242339+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:41:11.242339+0000","last_peered":"2024-03-23T23:41:11.242339+0000","last_clean":"2024-03-23T23:41:11.242339+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:41:11.242339+0000","last_undegraded":"2024-03-23T23:41:11.242339+0000","last_fullsized":"2024-03-23T23:41:11.242339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.241922+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:41:11.241922+0000","last_peered":"2024-03-23T23:41:11.241922+0000","last_clean":"2024-03-23T23:41:11.241922+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:41:11.241922+0000","last_undegraded":"2024-03-23T23:41:11.241922+0000","last_fullsized":"2024-03-23T23:41:11.241922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599316+0000","last_change":"2024-03-23T23:41:10.287013+0000","last_active":"2024-03-23T23:41:11.599316+0000","last_peered":"2024-03-23T23:41:11.599316+0000","last_clean":"2024-03-23T23:41:11.599316+0000","last_became_active":"2024-03-23T23:41:10.285062+0000","last_became_peered":"2024-03-23T23:41:10.285062+0000","last_unstale":"2024-03-23T23:41:11.599316+0000","last_undegraded":"2024-03-23T23:41:11.599316+0000","last_fullsized":"2024-03-23T23:41:11.599316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:01:51.764172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388255+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:41:11.388255+0000","last_peered":"2024-03-23T23:41:11.388255+0000","last_clean":"2024-03-23T23:41:11.388255+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:41:11.388255+0000","last_undegraded":"2024-03-23T23:41:11.388255+0000","last_fullsized":"2024-03-23T23:41:11.388255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599271+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:41:11.599271+0000","last_peered":"2024-03-23T23:41:11.599271+0000","last_clean":"2024-03-23T23:41:11.599271+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:41:11.599271+0000","last_undegraded":"2024-03-23T23:41:11.599271+0000","last_fullsized":"2024-03-23T23:41:11.599271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242498+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:41:11.242498+0000","last_peered":"2024-03-23T23:41:11.242498+0000","last_clean":"2024-03-23T23:41:11.242498+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:41:11.242498+0000","last_undegraded":"2024-03-23T23:41:11.242498+0000","last_fullsized":"2024-03-23T23:41:11.242498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600368+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:41:11.600368+0000","last_peered":"2024-03-23T23:41:11.600368+0000","last_clean":"2024-03-23T23:41:11.600368+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:41:11.600368+0000","last_undegraded":"2024-03-23T23:41:11.600368+0000","last_fullsized":"2024-03-23T23:41:11.600368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599535+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:41:11.599535+0000","last_peered":"2024-03-23T23:41:11.599535+0000","last_clean":"2024-03-23T23:41:11.599535+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:41:11.599535+0000","last_undegraded":"2024-03-23T23:41:11.599535+0000","last_fullsized":"2024-03-23T23:41:11.599535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600029+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:41:11.600029+0000","last_peered":"2024-03-23T23:41:11.600029+0000","last_clean":"2024-03-23T23:41:11.600029+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:41:11.600029+0000","last_undegraded":"2024-03-23T23:41:11.600029+0000","last_fullsized":"2024-03-23T23:41:11.600029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239953+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:41:11.239953+0000","last_peered":"2024-03-23T23:41:11.239953+0000","last_clean":"2024-03-23T23:41:11.239953+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:41:11.239953+0000","last_undegraded":"2024-03-23T23:41:11.239953+0000","last_fullsized":"2024-03-23T23:41:11.239953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388838+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:41:11.388838+0000","last_peered":"2024-03-23T23:41:11.388838+0000","last_clean":"2024-03-23T23:41:11.388838+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:41:11.388838+0000","last_undegraded":"2024-03-23T23:41:11.388838+0000","last_fullsized":"2024-03-23T23:41:11.388838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598814+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:41:11.598814+0000","last_peered":"2024-03-23T23:41:11.598814+0000","last_clean":"2024-03-23T23:41:11.598814+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:41:11.598814+0000","last_undegraded":"2024-03-23T23:41:11.598814+0000","last_fullsized":"2024-03-23T23:41:11.598814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6934,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":29,"num_read_kb":26,"num_write":172,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":20044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":438633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1032100,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:41:20.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T23:41:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 -- 172.21.15.99:0/1421809441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01006f0 msgr2=0x7f4df0100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 --2- 172.21.15.99:0/1421809441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01006f0 0x7f4df0100b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009900 tx=0x7f4ddc02bc30 comp rx=0 tx=0).stop 2024-03-23T23:41:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 -- 172.21.15.99:0/1421809441 shutdown_connections 2024-03-23T23:41:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 --2- 172.21.15.99:0/1421809441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df01010b0 0x7f4df0101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 --2- 172.21.15.99:0/1421809441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01006f0 0x7f4df0100b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 --2- 172.21.15.99:0/1421809441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df00ff990 0x7f4df00ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.652+0000 7f4df5000640 1 -- 172.21.15.99:0/1421809441 >> 172.21.15.99:0/1421809441 conn(0x7f4df00fb2c0 msgr2=0x7f4df00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.653+0000 7f4df5000640 1 -- 172.21.15.99:0/1421809441 shutdown_connections 2024-03-23T23:41:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.653+0000 7f4df5000640 1 -- 172.21.15.99:0/1421809441 wait complete. 2024-03-23T23:41:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.653+0000 7f4df5000640 1 Processor -- start 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 -- start start 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df01006f0 0x7f4df019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01010b0 0x7f4df01a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df010f5a0 con 0x7f4df01006f0 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df010f420 con 0x7f4df00ff990 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4df5000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df010f720 con 0x7f4df01010b0 2024-03-23T23:41:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4dede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4dede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42314/0 (socket says 172.21.15.99:42314) 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4dee800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01010b0 0x7f4df01a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4dede00640 1 -- 172.21.15.99:0/1020792125 learned_addr learned my addr 172.21.15.99:0/1020792125 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.654+0000 7f4ded400640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df01006f0 0x7f4df019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 -- 172.21.15.99:0/1020792125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01010b0 msgr2=0x7f4df01a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01010b0 0x7f4df01a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 -- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df01006f0 msgr2=0x7f4df019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df01006f0 0x7f4df019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 -- 172.21.15.99:0/1020792125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df01a17e0 con 0x7f4df00ff990 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4dede00640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4de000e9f0 tx=0x7f4de000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000cd80 con 0x7f4df00ff990 2024-03-23T23:41:20.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4de0010910 con 0x7f4df00ff990 2024-03-23T23:41:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df01a1ad0 con 0x7f4df00ff990 2024-03-23T23:41:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0018990 con 0x7f4df00ff990 2024-03-23T23:41:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.655+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df01a2040 con 0x7f4df00ff990 2024-03-23T23:41:20.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.657+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00688e0 con 0x7f4df00ff990 2024-03-23T23:41:20.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.658+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f4de0010ac0 con 0x7f4df00ff990 2024-03-23T23:41:20.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.660+0000 7f4de6a00640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 0x7f4dd40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:20.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.661+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4de00cecd0 con 0x7f4df00ff990 2024-03-23T23:41:20.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.661+0000 7f4ded400640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 0x7f4dd40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:20.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.661+0000 7f4ded400640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 0x7f4dd40a4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02c170 tx=0x7f4ddc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:20.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.664+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4de009b770 con 0x7f4df00ff990 2024-03-23T23:41:20.752 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:20 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2743034507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:20.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.913+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4df0061ba0 con 0x7f4dd40a2910 2024-03-23T23:41:20.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.921+0000 7f4de6a00640 1 -- 172.21.15.99:0/1020792125 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259001 (secure 0 0 0) 0x7f4df0061ba0 con 0x7f4dd40a2910 2024-03-23T23:41:20.924 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:41:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 msgr2=0x7f4dd40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 0x7f4dd40a4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc02c170 tx=0x7f4ddc02df90 comp rx=0 tx=0).stop 2024-03-23T23:41:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 msgr2=0x7f4df0199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4de000e9f0 tx=0x7f4de000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 shutdown_connections 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dd40a2910 0x7f4dd40a4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df01010b0 0x7f4df01a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4df01006f0 0x7f4df019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 --2- 172.21.15.99:0/1020792125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4df00ff990 0x7f4df0199bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 >> 172.21.15.99:0/1020792125 conn(0x7f4df00fb2c0 msgr2=0x7f4df00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 shutdown_connections 2024-03-23T23:41:20.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:20.926+0000 7f4df5000640 1 -- 172.21.15.99:0/1020792125 wait complete. 2024-03-23T23:41:20.953 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-03-23T23:41:20.727501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643369,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":185,"num_read_kb":198,"num_write":405,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":334,"ondisk_log_size":334,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381252,"kb_used_data":11300,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560572,"statfs":{"total":1151940427776,"available":1151550025728,"internally_reserved":0,"allocated":11571200,"data_stored":7145812,"data_compressed":110637,"data_compressed_allocated":143360,"data_compressed_original":1785856,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53001,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":59,"num_read_kb":89,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599311+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:41:11.599311+0000","last_peered":"2024-03-23T23:41:11.599311+0000","last_clean":"2024-03-23T23:41:11.599311+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:41:11.599311+0000","last_undegraded":"2024-03-23T23:41:11.599311+0000","last_fullsized":"2024-03-23T23:41:11.599311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599264+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:41:11.599264+0000","last_peered":"2024-03-23T23:41:11.599264+0000","last_clean":"2024-03-23T23:41:11.599264+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:41:11.599264+0000","last_undegraded":"2024-03-23T23:41:11.599264+0000","last_fullsized":"2024-03-23T23:41:11.599264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599453+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:41:11.599453+0000","last_peered":"2024-03-23T23:41:11.599453+0000","last_clean":"2024-03-23T23:41:11.599453+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:41:11.599453+0000","last_undegraded":"2024-03-23T23:41:11.599453+0000","last_fullsized":"2024-03-23T23:41:11.599453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599310+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:41:11.599310+0000","last_peered":"2024-03-23T23:41:11.599310+0000","last_clean":"2024-03-23T23:41:11.599310+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:41:11.599310+0000","last_undegraded":"2024-03-23T23:41:11.599310+0000","last_fullsized":"2024-03-23T23:41:11.599310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599270+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:41:11.599270+0000","last_peered":"2024-03-23T23:41:11.599270+0000","last_clean":"2024-03-23T23:41:11.599270+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:41:11.599270+0000","last_undegraded":"2024-03-23T23:41:11.599270+0000","last_fullsized":"2024-03-23T23:41:11.599270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599565+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:41:11.599565+0000","last_peered":"2024-03-23T23:41:11.599565+0000","last_clean":"2024-03-23T23:41:11.599565+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:41:11.599565+0000","last_undegraded":"2024-03-23T23:41:11.599565+0000","last_fullsized":"2024-03-23T23:41:11.599565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861675+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:41:11.861675+0000","last_peered":"2024-03-23T23:41:11.861675+0000","last_clean":"2024-03-23T23:41:11.861675+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:41:11.861675+0000","last_undegraded":"2024-03-23T23:41:11.861675+0000","last_fullsized":"2024-03-23T23:41:11.861675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599074+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:41:11.599074+0000","last_peered":"2024-03-23T23:41:11.599074+0000","last_clean":"2024-03-23T23:41:11.599074+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:41:11.599074+0000","last_undegraded":"2024-03-23T23:41:11.599074+0000","last_fullsized":"2024-03-23T23:41:11.599074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240173+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:41:11.240173+0000","last_peered":"2024-03-23T23:41:11.240173+0000","last_clean":"2024-03-23T23:41:11.240173+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:41:11.240173+0000","last_undegraded":"2024-03-23T23:41:11.240173+0000","last_fullsized":"2024-03-23T23:41:11.240173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599876+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:41:11.599876+0000","last_peered":"2024-03-23T23:41:11.599876+0000","last_clean":"2024-03-23T23:41:11.599876+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:41:11.599876+0000","last_undegraded":"2024-03-23T23:41:11.599876+0000","last_fullsized":"2024-03-23T23:41:11.599876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.862021+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:41:11.862021+0000","last_peered":"2024-03-23T23:41:11.862021+0000","last_clean":"2024-03-23T23:41:11.862021+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:41:11.862021+0000","last_undegraded":"2024-03-23T23:41:11.862021+0000","last_fullsized":"2024-03-23T23:41:11.862021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599195+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:41:11.599195+0000","last_peered":"2024-03-23T23:41:11.599195+0000","last_clean":"2024-03-23T23:41:11.599195+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:41:11.599195+0000","last_undegraded":"2024-03-23T23:41:11.599195+0000","last_fullsized":"2024-03-23T23:41:11.599195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388827+0000","last_change":"2024-03-23T23:41:10.276300+0000","last_active":"2024-03-23T23:41:11.388827+0000","last_peered":"2024-03-23T23:41:11.388827+0000","last_clean":"2024-03-23T23:41:11.388827+0000","last_became_active":"2024-03-23T23:41:10.275762+0000","last_became_peered":"2024-03-23T23:41:10.275762+0000","last_unstale":"2024-03-23T23:41:11.388827+0000","last_undegraded":"2024-03-23T23:41:11.388827+0000","last_fullsized":"2024-03-23T23:41:11.388827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:44:38.705104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598731+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:41:11.598731+0000","last_peered":"2024-03-23T23:41:11.598731+0000","last_clean":"2024-03-23T23:41:11.598731+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:41:11.598731+0000","last_undegraded":"2024-03-23T23:41:11.598731+0000","last_fullsized":"2024-03-23T23:41:11.598731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598627+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:41:11.598627+0000","last_peered":"2024-03-23T23:41:11.598627+0000","last_clean":"2024-03-23T23:41:11.598627+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:41:11.598627+0000","last_undegraded":"2024-03-23T23:41:11.598627+0000","last_fullsized":"2024-03-23T23:41:11.598627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239607+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:41:11.239607+0000","last_peered":"2024-03-23T23:41:11.239607+0000","last_clean":"2024-03-23T23:41:11.239607+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:41:11.239607+0000","last_undegraded":"2024-03-23T23:41:11.239607+0000","last_fullsized":"2024-03-23T23:41:11.239607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599325+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:41:11.599325+0000","last_peered":"2024-03-23T23:41:11.599325+0000","last_clean":"2024-03-23T23:41:11.599325+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:41:11.599325+0000","last_undegraded":"2024-03-23T23:41:11.599325+0000","last_fullsized":"2024-03-23T23:41:11.599325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.241446+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:41:11.241446+0000","last_peered":"2024-03-23T23:41:11.241446+0000","last_clean":"2024-03-23T23:41:11.241446+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:41:11.241446+0000","last_undegraded":"2024-03-23T23:41:11.241446+0000","last_fullsized":"2024-03-23T23:41:11.241446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242186+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:41:11.242186+0000","last_peered":"2024-03-23T23:41:11.242186+0000","last_clean":"2024-03-23T23:41:11.242186+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:41:11.242186+0000","last_undegraded":"2024-03-23T23:41:11.242186+0000","last_fullsized":"2024-03-23T23:41:11.242186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.291026+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:41:17.291026+0000","last_peered":"2024-03-23T23:41:17.291026+0000","last_clean":"2024-03-23T23:41:17.291026+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:41:17.291026+0000","last_undegraded":"2024-03-23T23:41:17.291026+0000","last_fullsized":"2024-03-23T23:41:17.291026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599668+0000","last_change":"2024-03-23T23:41:10.274414+0000","last_active":"2024-03-23T23:41:11.599668+0000","last_peered":"2024-03-23T23:41:11.599668+0000","last_clean":"2024-03-23T23:41:11.599668+0000","last_became_active":"2024-03-23T23:41:10.272770+0000","last_became_peered":"2024-03-23T23:41:10.272770+0000","last_unstale":"2024-03-23T23:41:11.599668+0000","last_undegraded":"2024-03-23T23:41:11.599668+0000","last_fullsized":"2024-03-23T23:41:11.599668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:02:51.844280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600320+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:41:11.600320+0000","last_peered":"2024-03-23T23:41:11.600320+0000","last_clean":"2024-03-23T23:41:11.600320+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:41:11.600320+0000","last_undegraded":"2024-03-23T23:41:11.600320+0000","last_fullsized":"2024-03-23T23:41:11.600320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599260+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:41:11.599260+0000","last_peered":"2024-03-23T23:41:11.599260+0000","last_clean":"2024-03-23T23:41:11.599260+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:41:11.599260+0000","last_undegraded":"2024-03-23T23:41:11.599260+0000","last_fullsized":"2024-03-23T23:41:11.599260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.288093+0000","last_change":"2024-03-23T23:41:09.245775+0000","last_active":"2024-03-23T23:41:17.288093+0000","last_peered":"2024-03-23T23:41:17.288093+0000","last_clean":"2024-03-23T23:41:17.288093+0000","last_became_active":"2024-03-23T23:41:09.245406+0000","last_became_peered":"2024-03-23T23:41:09.245406+0000","last_unstale":"2024-03-23T23:41:17.288093+0000","last_undegraded":"2024-03-23T23:41:17.288093+0000","last_fullsized":"2024-03-23T23:41:17.288093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:31:40.079757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239669+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:41:11.239669+0000","last_peered":"2024-03-23T23:41:11.239669+0000","last_clean":"2024-03-23T23:41:11.239669+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:41:11.239669+0000","last_undegraded":"2024-03-23T23:41:11.239669+0000","last_fullsized":"2024-03-23T23:41:11.239669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.744693+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:41:16.744693+0000","last_peered":"2024-03-23T23:41:16.744693+0000","last_clean":"2024-03-23T23:41:16.744693+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:41:16.744693+0000","last_undegraded":"2024-03-23T23:41:16.744693+0000","last_fullsized":"2024-03-23T23:41:16.744693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599668+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:41:11.599668+0000","last_peered":"2024-03-23T23:41:11.599668+0000","last_clean":"2024-03-23T23:41:11.599668+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:41:11.599668+0000","last_undegraded":"2024-03-23T23:41:11.599668+0000","last_fullsized":"2024-03-23T23:41:11.599668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.289187+0000","last_change":"2024-03-23T23:41:09.251958+0000","last_active":"2024-03-23T23:41:17.289187+0000","last_peered":"2024-03-23T23:41:17.289187+0000","last_clean":"2024-03-23T23:41:17.289187+0000","last_became_active":"2024-03-23T23:41:09.251609+0000","last_became_peered":"2024-03-23T23:41:09.251609+0000","last_unstale":"2024-03-23T23:41:17.289187+0000","last_undegraded":"2024-03-23T23:41:17.289187+0000","last_fullsized":"2024-03-23T23:41:17.289187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:46:43.853473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599652+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:41:11.599652+0000","last_peered":"2024-03-23T23:41:11.599652+0000","last_clean":"2024-03-23T23:41:11.599652+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:41:11.599652+0000","last_undegraded":"2024-03-23T23:41:11.599652+0000","last_fullsized":"2024-03-23T23:41:11.599652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.730024+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:41:15.730024+0000","last_peered":"2024-03-23T23:41:15.730024+0000","last_clean":"2024-03-23T23:41:15.730024+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:41:15.730024+0000","last_undegraded":"2024-03-23T23:41:15.730024+0000","last_fullsized":"2024-03-23T23:41:15.730024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599711+0000","last_change":"2024-03-23T23:41:10.270493+0000","last_active":"2024-03-23T23:41:11.599711+0000","last_peered":"2024-03-23T23:41:11.599711+0000","last_clean":"2024-03-23T23:41:11.599711+0000","last_became_active":"2024-03-23T23:41:10.270141+0000","last_became_peered":"2024-03-23T23:41:10.270141+0000","last_unstale":"2024-03-23T23:41:11.599711+0000","last_undegraded":"2024-03-23T23:41:11.599711+0000","last_fullsized":"2024-03-23T23:41:11.599711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:33:55.277951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.729253+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:41:14.729253+0000","last_peered":"2024-03-23T23:41:14.729253+0000","last_clean":"2024-03-23T23:41:14.729253+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:41:14.729253+0000","last_undegraded":"2024-03-23T23:41:14.729253+0000","last_fullsized":"2024-03-23T23:41:14.729253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598896+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:41:11.598896+0000","last_peered":"2024-03-23T23:41:11.598896+0000","last_clean":"2024-03-23T23:41:11.598896+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:41:11.598896+0000","last_undegraded":"2024-03-23T23:41:11.598896+0000","last_fullsized":"2024-03-23T23:41:11.598896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.745257+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:41:16.745257+0000","last_peered":"2024-03-23T23:41:16.745257+0000","last_clean":"2024-03-23T23:41:16.745257+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:41:16.745257+0000","last_undegraded":"2024-03-23T23:41:16.745257+0000","last_fullsized":"2024-03-23T23:41:16.745257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240694+0000","last_change":"2024-03-23T23:41:10.271143+0000","last_active":"2024-03-23T23:41:11.240694+0000","last_peered":"2024-03-23T23:41:11.240694+0000","last_clean":"2024-03-23T23:41:11.240694+0000","last_became_active":"2024-03-23T23:41:10.270678+0000","last_became_peered":"2024-03-23T23:41:10.270678+0000","last_unstale":"2024-03-23T23:41:11.240694+0000","last_undegraded":"2024-03-23T23:41:11.240694+0000","last_fullsized":"2024-03-23T23:41:11.240694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:01:36.290851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:20.293371+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:41:20.293371+0000","last_peered":"2024-03-23T23:41:20.293371+0000","last_clean":"2024-03-23T23:41:20.293371+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:41:20.293371+0000","last_undegraded":"2024-03-23T23:41:20.293371+0000","last_fullsized":"2024-03-23T23:41:20.293371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240047+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:41:11.240047+0000","last_peered":"2024-03-23T23:41:11.240047+0000","last_clean":"2024-03-23T23:41:11.240047+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:41:11.240047+0000","last_undegraded":"2024-03-23T23:41:11.240047+0000","last_fullsized":"2024-03-23T23:41:11.240047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.744708+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:41:16.744708+0000","last_peered":"2024-03-23T23:41:16.744708+0000","last_clean":"2024-03-23T23:41:16.744708+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:41:16.744708+0000","last_undegraded":"2024-03-23T23:41:16.744708+0000","last_fullsized":"2024-03-23T23:41:16.744708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599983+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:41:11.599983+0000","last_peered":"2024-03-23T23:41:11.599983+0000","last_clean":"2024-03-23T23:41:11.599983+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:41:11.599983+0000","last_undegraded":"2024-03-23T23:41:11.599983+0000","last_fullsized":"2024-03-23T23:41:11.599983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.729693+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:41:15.729693+0000","last_peered":"2024-03-23T23:41:15.729693+0000","last_clean":"2024-03-23T23:41:15.729693+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:41:15.729693+0000","last_undegraded":"2024-03-23T23:41:15.729693+0000","last_fullsized":"2024-03-23T23:41:15.729693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239975+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:41:11.239975+0000","last_peered":"2024-03-23T23:41:11.239975+0000","last_clean":"2024-03-23T23:41:11.239975+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:41:11.239975+0000","last_undegraded":"2024-03-23T23:41:11.239975+0000","last_fullsized":"2024-03-23T23:41:11.239975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'23","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:20.296511+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:41:20.296511+0000","last_peered":"2024-03-23T23:41:20.296511+0000","last_clean":"2024-03-23T23:41:20.296511+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:41:20.296511+0000","last_undegraded":"2024-03-23T23:41:20.296511+0000","last_fullsized":"2024-03-23T23:41:20.296511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45563,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":50,"num_write":25,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600107+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:41:11.600107+0000","last_peered":"2024-03-23T23:41:11.600107+0000","last_clean":"2024-03-23T23:41:11.600107+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:41:11.600107+0000","last_undegraded":"2024-03-23T23:41:11.600107+0000","last_fullsized":"2024-03-23T23:41:11.600107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.287599+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:41:17.287599+0000","last_peered":"2024-03-23T23:41:17.287599+0000","last_clean":"2024-03-23T23:41:17.287599+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:41:17.287599+0000","last_undegraded":"2024-03-23T23:41:17.287599+0000","last_fullsized":"2024-03-23T23:41:17.287599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388678+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:41:11.388678+0000","last_peered":"2024-03-23T23:41:11.388678+0000","last_clean":"2024-03-23T23:41:11.388678+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:41:11.388678+0000","last_undegraded":"2024-03-23T23:41:11.388678+0000","last_fullsized":"2024-03-23T23:41:11.388678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748360+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:41:16.748360+0000","last_peered":"2024-03-23T23:41:16.748360+0000","last_clean":"2024-03-23T23:41:16.748360+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:41:16.748360+0000","last_undegraded":"2024-03-23T23:41:16.748360+0000","last_fullsized":"2024-03-23T23:41:16.748360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861891+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:41:11.861891+0000","last_peered":"2024-03-23T23:41:11.861891+0000","last_clean":"2024-03-23T23:41:11.861891+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:41:11.861891+0000","last_undegraded":"2024-03-23T23:41:11.861891+0000","last_fullsized":"2024-03-23T23:41:11.861891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.291714+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:41:17.291714+0000","last_peered":"2024-03-23T23:41:17.291714+0000","last_clean":"2024-03-23T23:41:17.291714+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:41:17.291714+0000","last_undegraded":"2024-03-23T23:41:17.291714+0000","last_fullsized":"2024-03-23T23:41:17.291714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861626+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:41:11.861626+0000","last_peered":"2024-03-23T23:41:11.861626+0000","last_clean":"2024-03-23T23:41:11.861626+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:41:11.861626+0000","last_undegraded":"2024-03-23T23:41:11.861626+0000","last_fullsized":"2024-03-23T23:41:11.861626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.747007+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:41:16.747007+0000","last_peered":"2024-03-23T23:41:16.747007+0000","last_clean":"2024-03-23T23:41:16.747007+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:41:16.747007+0000","last_undegraded":"2024-03-23T23:41:16.747007+0000","last_fullsized":"2024-03-23T23:41:16.747007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599614+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:41:11.599614+0000","last_peered":"2024-03-23T23:41:11.599614+0000","last_clean":"2024-03-23T23:41:11.599614+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:41:11.599614+0000","last_undegraded":"2024-03-23T23:41:11.599614+0000","last_fullsized":"2024-03-23T23:41:11.599614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.717461+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:41:15.717461+0000","last_peered":"2024-03-23T23:41:15.717461+0000","last_clean":"2024-03-23T23:41:15.717461+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:41:15.717461+0000","last_undegraded":"2024-03-23T23:41:15.717461+0000","last_fullsized":"2024-03-23T23:41:15.717461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599183+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:41:11.599183+0000","last_peered":"2024-03-23T23:41:11.599183+0000","last_clean":"2024-03-23T23:41:11.599183+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:41:11.599183+0000","last_undegraded":"2024-03-23T23:41:11.599183+0000","last_fullsized":"2024-03-23T23:41:11.599183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.240749+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:41:11.240749+0000","last_peered":"2024-03-23T23:41:11.240749+0000","last_clean":"2024-03-23T23:41:11.240749+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:41:11.240749+0000","last_undegraded":"2024-03-23T23:41:11.240749+0000","last_fullsized":"2024-03-23T23:41:11.240749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599803+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:41:11.599803+0000","last_peered":"2024-03-23T23:41:11.599803+0000","last_clean":"2024-03-23T23:41:11.599803+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:41:11.599803+0000","last_undegraded":"2024-03-23T23:41:11.599803+0000","last_fullsized":"2024-03-23T23:41:11.599803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:18.293651+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:41:18.293651+0000","last_peered":"2024-03-23T23:41:18.293651+0000","last_clean":"2024-03-23T23:41:18.293651+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:41:18.293651+0000","last_undegraded":"2024-03-23T23:41:18.293651+0000","last_fullsized":"2024-03-23T23:41:18.293651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599590+0000","last_change":"2024-03-23T23:41:10.286442+0000","last_active":"2024-03-23T23:41:11.599590+0000","last_peered":"2024-03-23T23:41:11.599590+0000","last_clean":"2024-03-23T23:41:11.599590+0000","last_became_active":"2024-03-23T23:41:10.286085+0000","last_became_peered":"2024-03-23T23:41:10.286085+0000","last_unstale":"2024-03-23T23:41:11.599590+0000","last_undegraded":"2024-03-23T23:41:11.599590+0000","last_fullsized":"2024-03-23T23:41:11.599590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:58.730743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599539+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:41:11.599539+0000","last_peered":"2024-03-23T23:41:11.599539+0000","last_clean":"2024-03-23T23:41:11.599539+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:41:11.599539+0000","last_undegraded":"2024-03-23T23:41:11.599539+0000","last_fullsized":"2024-03-23T23:41:11.599539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.729834+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:41:14.729834+0000","last_peered":"2024-03-23T23:41:14.729834+0000","last_clean":"2024-03-23T23:41:14.729834+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:41:14.729834+0000","last_undegraded":"2024-03-23T23:41:14.729834+0000","last_fullsized":"2024-03-23T23:41:14.729834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599352+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:41:11.599352+0000","last_peered":"2024-03-23T23:41:11.599352+0000","last_clean":"2024-03-23T23:41:11.599352+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:41:11.599352+0000","last_undegraded":"2024-03-23T23:41:11.599352+0000","last_fullsized":"2024-03-23T23:41:11.599352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.288109+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:41:17.288109+0000","last_peered":"2024-03-23T23:41:17.288109+0000","last_clean":"2024-03-23T23:41:17.288109+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:41:17.288109+0000","last_undegraded":"2024-03-23T23:41:17.288109+0000","last_fullsized":"2024-03-23T23:41:17.288109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861624+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:41:11.861624+0000","last_peered":"2024-03-23T23:41:11.861624+0000","last_clean":"2024-03-23T23:41:11.861624+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:41:11.861624+0000","last_undegraded":"2024-03-23T23:41:11.861624+0000","last_fullsized":"2024-03-23T23:41:11.861624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.387944+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:41:11.387944+0000","last_peered":"2024-03-23T23:41:11.387944+0000","last_clean":"2024-03-23T23:41:11.387944+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:41:11.387944+0000","last_undegraded":"2024-03-23T23:41:11.387944+0000","last_fullsized":"2024-03-23T23:41:11.387944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:14.722487+0000","last_change":"2024-03-23T23:41:09.252221+0000","last_active":"2024-03-23T23:41:14.722487+0000","last_peered":"2024-03-23T23:41:14.722487+0000","last_clean":"2024-03-23T23:41:14.722487+0000","last_became_active":"2024-03-23T23:41:09.251930+0000","last_became_peered":"2024-03-23T23:41:09.251930+0000","last_unstale":"2024-03-23T23:41:14.722487+0000","last_undegraded":"2024-03-23T23:41:14.722487+0000","last_fullsized":"2024-03-23T23:41:14.722487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:16:31.262380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748873+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:41:16.748873+0000","last_peered":"2024-03-23T23:41:16.748873+0000","last_clean":"2024-03-23T23:41:16.748873+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:41:16.748873+0000","last_undegraded":"2024-03-23T23:41:16.748873+0000","last_fullsized":"2024-03-23T23:41:16.748873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388365+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:41:11.388365+0000","last_peered":"2024-03-23T23:41:11.388365+0000","last_clean":"2024-03-23T23:41:11.388365+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:41:11.388365+0000","last_undegraded":"2024-03-23T23:41:11.388365+0000","last_fullsized":"2024-03-23T23:41:11.388365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.735482+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:41:15.735482+0000","last_peered":"2024-03-23T23:41:15.735482+0000","last_clean":"2024-03-23T23:41:15.735482+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:41:15.735482+0000","last_undegraded":"2024-03-23T23:41:15.735482+0000","last_fullsized":"2024-03-23T23:41:15.735482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600480+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:41:11.600480+0000","last_peered":"2024-03-23T23:41:11.600480+0000","last_clean":"2024-03-23T23:41:11.600480+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:41:11.600480+0000","last_undegraded":"2024-03-23T23:41:11.600480+0000","last_fullsized":"2024-03-23T23:41:11.600480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.747226+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:41:16.747226+0000","last_peered":"2024-03-23T23:41:16.747226+0000","last_clean":"2024-03-23T23:41:16.747226+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:41:16.747226+0000","last_undegraded":"2024-03-23T23:41:16.747226+0000","last_fullsized":"2024-03-23T23:41:16.747226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599428+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:41:11.599428+0000","last_peered":"2024-03-23T23:41:11.599428+0000","last_clean":"2024-03-23T23:41:11.599428+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:41:11.599428+0000","last_undegraded":"2024-03-23T23:41:11.599428+0000","last_fullsized":"2024-03-23T23:41:11.599428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.706709+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:41:13.706709+0000","last_peered":"2024-03-23T23:41:13.706709+0000","last_clean":"2024-03-23T23:41:13.706709+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:41:13.706709+0000","last_undegraded":"2024-03-23T23:41:13.706709+0000","last_fullsized":"2024-03-23T23:41:13.706709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598870+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:41:11.598870+0000","last_peered":"2024-03-23T23:41:11.598870+0000","last_clean":"2024-03-23T23:41:11.598870+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:41:11.598870+0000","last_undegraded":"2024-03-23T23:41:11.598870+0000","last_fullsized":"2024-03-23T23:41:11.598870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748574+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:41:16.748574+0000","last_peered":"2024-03-23T23:41:16.748574+0000","last_clean":"2024-03-23T23:41:16.748574+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:41:16.748574+0000","last_undegraded":"2024-03-23T23:41:16.748574+0000","last_fullsized":"2024-03-23T23:41:16.748574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598964+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:41:11.598964+0000","last_peered":"2024-03-23T23:41:11.598964+0000","last_clean":"2024-03-23T23:41:11.598964+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:41:11.598964+0000","last_undegraded":"2024-03-23T23:41:11.598964+0000","last_fullsized":"2024-03-23T23:41:11.598964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:15.732690+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:41:15.732690+0000","last_peered":"2024-03-23T23:41:15.732690+0000","last_clean":"2024-03-23T23:41:15.732690+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:41:15.732690+0000","last_undegraded":"2024-03-23T23:41:15.732690+0000","last_fullsized":"2024-03-23T23:41:15.732690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599932+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:41:11.599932+0000","last_peered":"2024-03-23T23:41:11.599932+0000","last_clean":"2024-03-23T23:41:11.599932+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:41:11.599932+0000","last_undegraded":"2024-03-23T23:41:11.599932+0000","last_fullsized":"2024-03-23T23:41:11.599932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:13.707796+0000","last_change":"2024-03-23T23:41:09.247532+0000","last_active":"2024-03-23T23:41:13.707796+0000","last_peered":"2024-03-23T23:41:13.707796+0000","last_clean":"2024-03-23T23:41:13.707796+0000","last_became_active":"2024-03-23T23:41:09.245141+0000","last_became_peered":"2024-03-23T23:41:09.245141+0000","last_unstale":"2024-03-23T23:41:13.707796+0000","last_undegraded":"2024-03-23T23:41:13.707796+0000","last_fullsized":"2024-03-23T23:41:13.707796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:05:42.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861961+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:41:11.861961+0000","last_peered":"2024-03-23T23:41:11.861961+0000","last_clean":"2024-03-23T23:41:11.861961+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:41:11.861961+0000","last_undegraded":"2024-03-23T23:41:11.861961+0000","last_fullsized":"2024-03-23T23:41:11.861961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:17.291428+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:41:17.291428+0000","last_peered":"2024-03-23T23:41:17.291428+0000","last_clean":"2024-03-23T23:41:17.291428+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:41:17.291428+0000","last_undegraded":"2024-03-23T23:41:17.291428+0000","last_fullsized":"2024-03-23T23:41:17.291428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.862038+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:41:11.862038+0000","last_peered":"2024-03-23T23:41:11.862038+0000","last_clean":"2024-03-23T23:41:11.862038+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:41:11.862038+0000","last_undegraded":"2024-03-23T23:41:11.862038+0000","last_fullsized":"2024-03-23T23:41:11.862038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748382+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:41:16.748382+0000","last_peered":"2024-03-23T23:41:16.748382+0000","last_clean":"2024-03-23T23:41:16.748382+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:41:16.748382+0000","last_undegraded":"2024-03-23T23:41:16.748382+0000","last_fullsized":"2024-03-23T23:41:16.748382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.862050+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:41:11.862050+0000","last_peered":"2024-03-23T23:41:11.862050+0000","last_clean":"2024-03-23T23:41:11.862050+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:41:11.862050+0000","last_undegraded":"2024-03-23T23:41:11.862050+0000","last_fullsized":"2024-03-23T23:41:11.862050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:16.748076+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:41:16.748076+0000","last_peered":"2024-03-23T23:41:16.748076+0000","last_clean":"2024-03-23T23:41:16.748076+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:41:16.748076+0000","last_undegraded":"2024-03-23T23:41:16.748076+0000","last_fullsized":"2024-03-23T23:41:16.748076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.861968+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:41:11.861968+0000","last_peered":"2024-03-23T23:41:11.861968+0000","last_clean":"2024-03-23T23:41:11.861968+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:41:11.861968+0000","last_undegraded":"2024-03-23T23:41:11.861968+0000","last_fullsized":"2024-03-23T23:41:11.861968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599238+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:41:11.599238+0000","last_peered":"2024-03-23T23:41:11.599238+0000","last_clean":"2024-03-23T23:41:11.599238+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:41:11.599238+0000","last_undegraded":"2024-03-23T23:41:11.599238+0000","last_fullsized":"2024-03-23T23:41:11.599238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242339+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:41:11.242339+0000","last_peered":"2024-03-23T23:41:11.242339+0000","last_clean":"2024-03-23T23:41:11.242339+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:41:11.242339+0000","last_undegraded":"2024-03-23T23:41:11.242339+0000","last_fullsized":"2024-03-23T23:41:11.242339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.241922+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:41:11.241922+0000","last_peered":"2024-03-23T23:41:11.241922+0000","last_clean":"2024-03-23T23:41:11.241922+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:41:11.241922+0000","last_undegraded":"2024-03-23T23:41:11.241922+0000","last_fullsized":"2024-03-23T23:41:11.241922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599316+0000","last_change":"2024-03-23T23:41:10.287013+0000","last_active":"2024-03-23T23:41:11.599316+0000","last_peered":"2024-03-23T23:41:11.599316+0000","last_clean":"2024-03-23T23:41:11.599316+0000","last_became_active":"2024-03-23T23:41:10.285062+0000","last_became_peered":"2024-03-23T23:41:10.285062+0000","last_unstale":"2024-03-23T23:41:11.599316+0000","last_undegraded":"2024-03-23T23:41:11.599316+0000","last_fullsized":"2024-03-23T23:41:11.599316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:01:51.764172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388255+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:41:11.388255+0000","last_peered":"2024-03-23T23:41:11.388255+0000","last_clean":"2024-03-23T23:41:11.388255+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:41:11.388255+0000","last_undegraded":"2024-03-23T23:41:11.388255+0000","last_fullsized":"2024-03-23T23:41:11.388255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599271+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:41:11.599271+0000","last_peered":"2024-03-23T23:41:11.599271+0000","last_clean":"2024-03-23T23:41:11.599271+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:41:11.599271+0000","last_undegraded":"2024-03-23T23:41:11.599271+0000","last_fullsized":"2024-03-23T23:41:11.599271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.242498+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:41:11.242498+0000","last_peered":"2024-03-23T23:41:11.242498+0000","last_clean":"2024-03-23T23:41:11.242498+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:41:11.242498+0000","last_undegraded":"2024-03-23T23:41:11.242498+0000","last_fullsized":"2024-03-23T23:41:11.242498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600368+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:41:11.600368+0000","last_peered":"2024-03-23T23:41:11.600368+0000","last_clean":"2024-03-23T23:41:11.600368+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:41:11.600368+0000","last_undegraded":"2024-03-23T23:41:11.600368+0000","last_fullsized":"2024-03-23T23:41:11.600368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.599535+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:41:11.599535+0000","last_peered":"2024-03-23T23:41:11.599535+0000","last_clean":"2024-03-23T23:41:11.599535+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:41:11.599535+0000","last_undegraded":"2024-03-23T23:41:11.599535+0000","last_fullsized":"2024-03-23T23:41:11.599535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.600029+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:41:11.600029+0000","last_peered":"2024-03-23T23:41:11.600029+0000","last_clean":"2024-03-23T23:41:11.600029+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:41:11.600029+0000","last_undegraded":"2024-03-23T23:41:11.600029+0000","last_fullsized":"2024-03-23T23:41:11.600029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.239953+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:41:11.239953+0000","last_peered":"2024-03-23T23:41:11.239953+0000","last_clean":"2024-03-23T23:41:11.239953+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:41:11.239953+0000","last_undegraded":"2024-03-23T23:41:11.239953+0000","last_fullsized":"2024-03-23T23:41:11.239953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.388838+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:41:11.388838+0000","last_peered":"2024-03-23T23:41:11.388838+0000","last_clean":"2024-03-23T23:41:11.388838+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:41:11.388838+0000","last_undegraded":"2024-03-23T23:41:11.388838+0000","last_fullsized":"2024-03-23T23:41:11.388838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T23:41:11.598814+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:41:11.598814+0000","last_peered":"2024-03-23T23:41:11.598814+0000","last_clean":"2024-03-23T23:41:11.598814+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:41:11.598814+0000","last_undegraded":"2024-03-23T23:41:11.598814+0000","last_fullsized":"2024-03-23T23:41:11.598814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":52767,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":59,"num_read_kb":89,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":113180,"data_compressed":3354,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1076696,"data_compressed":37438,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1032930,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":438633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":486447,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":439185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1032100,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":47926,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":47690,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:41:20.958 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T23:41:20.958 INFO:tasks.cephadm:Setting up client nodes... 2024-03-23T23:41:20.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T23:41:21.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:20 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2743034507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:21.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:20 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2743034507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:21.456 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:21.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:21 smithi099 ceph-mon[31715]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:21.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:21 smithi099 ceph-mon[31715]: pgmap v322: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-23T23:41:21.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:21 smithi099 ceph-mon[31715]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:21 smithi149 ceph-mon[36083]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:21 smithi149 ceph-mon[36083]: pgmap v322: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-23T23:41:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:21 smithi149 ceph-mon[36083]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:21 smithi174 ceph-mon[37002]: from='client.24724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:21 smithi174 ceph-mon[37002]: pgmap v322: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-23T23:41:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:21 smithi174 ceph-mon[37002]: from='client.24707 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.897+0000 7f48d5800700 1 -- 172.21.15.99:0/2083445141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 msgr2=0x7f48d0103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.897+0000 7f48d5800700 1 --2- 172.21.15.99:0/2083445141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d0103850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009990 tx=0x7f48bc009ca0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 -- 172.21.15.99:0/2083445141 shutdown_connections 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 --2- 172.21.15.99:0/2083445141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d0103d90 0x7f48d010a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 --2- 172.21.15.99:0/2083445141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d01021f0 0x7f48d0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 --2- 172.21.15.99:0/2083445141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d0103850 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 -- 172.21.15.99:0/2083445141 >> 172.21.15.99:0/2083445141 conn(0x7f48d00fd770 msgr2=0x7f48d00ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 -- 172.21.15.99:0/2083445141 shutdown_connections 2024-03-23T23:41:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.898+0000 7f48d5800700 1 -- 172.21.15.99:0/2083445141 wait complete. 2024-03-23T23:41:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 Processor -- start 2024-03-23T23:41:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 -- start start 2024-03-23T23:41:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d01021f0 0x7f48d019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 0x7f48d01a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d010de10 con 0x7f48d01021f0 2024-03-23T23:41:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d010dc90 con 0x7f48d0103d90 2024-03-23T23:41:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.899+0000 7f48d5800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d010df90 con 0x7f48d01033f0 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48cf400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 0x7f48d01a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48cea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d01021f0 0x7f48d019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:36172/0 (socket says 172.21.15.99:36172) 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48cf400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 0x7f48d01a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42336/0 (socket says 172.21.15.99:42336) 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 -- 172.21.15.99:0/1089524764 learned_addr learned my addr 172.21.15.99:0/1089524764 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 -- 172.21.15.99:0/1089524764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 msgr2=0x7f48d01a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 0x7f48d01a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 -- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d01021f0 msgr2=0x7f48d019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d01021f0 0x7f48d019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.900+0000 7f48ce000700 1 -- 172.21.15.99:0/1089524764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d01a21e0 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48ce000700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b820 tx=0x7f48bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc027070 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48bc00fd70 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc017650 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01a2470 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.901+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01a2920 con 0x7f48d01033f0 2024-03-23T23:41:23.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.902+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d00506a0 con 0x7f48d01033f0 2024-03-23T23:41:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.904+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f48bc02a070 con 0x7f48d01033f0 2024-03-23T23:41:23.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.906+0000 7f48c7400700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 0x7f48b40a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:23.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.906+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48bc0cfa20 con 0x7f48d01033f0 2024-03-23T23:41:23.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.906+0000 7f48cea00700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 0x7f48b40a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:23.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.907+0000 7f48cea00700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 0x7f48b40a4f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48d0103250 tx=0x7f48c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:23.909+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f48bc05e740 con 0x7f48d01033f0 2024-03-23T23:41:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:23 smithi099 ceph-mon[31715]: pgmap v323: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-03-23T23:41:24.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:23 smithi149 ceph-mon[36083]: pgmap v323: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-03-23T23:41:24.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:23 smithi174 ceph-mon[37002]: pgmap v323: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-03-23T23:41:24.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.252+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f48d01a2cc0 con 0x7f48d01033f0 2024-03-23T23:41:24.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.260+0000 7f48c7400700 1 -- 172.21.15.99:0/1089524764 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f48bc013070 con 0x7f48d01033f0 2024-03-23T23:41:24.260 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-03-23T23:41:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQCkaP9l+7ksDxAA5ryvUVkZ0GSSSWGMYpwB/A== 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 msgr2=0x7f48b40a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 0x7f48b40a4f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f48d0103250 tx=0x7f48c0009260 comp rx=0 tx=0).stop 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 msgr2=0x7f48d019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b820 tx=0x7f48bc00f690 comp rx=0 tx=0).stop 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 shutdown_connections 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48b40a2a50 0x7f48b40a4f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:24.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0103d90 0x7f48d01a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48d01021f0 0x7f48d019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 --2- 172.21.15.99:0/1089524764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48d01033f0 0x7f48d019aac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 >> 172.21.15.99:0/1089524764 conn(0x7f48d00fd770 msgr2=0x7f48d00743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 shutdown_connections 2024-03-23T23:41:24.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:24.264+0000 7f48d5800700 1 -- 172.21.15.99:0/1089524764 wait complete. 2024-03-23T23:41:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:24 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1089524764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:24 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:24 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T23:41:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:24 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1089524764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:24 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:24 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T23:41:25.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:24 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1089524764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:24 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T23:41:25.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:24 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T23:41:25.212 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:41:25.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-23T23:41:25.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T23:41:25.352 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T23:41:25.352 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-23T23:41:25.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-23T23:41:25.869 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:26.031 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:25 smithi099 ceph-mon[31715]: pgmap v324: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-23T23:41:26.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:25 smithi149 ceph-mon[36083]: pgmap v324: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-23T23:41:26.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:25 smithi174 ceph-mon[37002]: pgmap v324: 97 pgs: 97 active+clean; 628 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-03-23T23:41:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:27 smithi099 ceph-mon[31715]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T23:41:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:27 smithi149 ceph-mon[36083]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T23:41:28.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:27 smithi174 ceph-mon[37002]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 24 op/s 2024-03-23T23:41:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 -- 172.21.15.99:0/2532291639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 msgr2=0x7fc68c108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:28.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 --2- 172.21.15.99:0/2532291639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c108fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009990 tx=0x7fc67c009ca0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 -- 172.21.15.99:0/2532291639 shutdown_connections 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 --2- 172.21.15.99:0/2532291639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 --2- 172.21.15.99:0/2532291639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 0x7fc68c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 --2- 172.21.15.99:0/2532291639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c108fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 -- 172.21.15.99:0/2532291639 >> 172.21.15.99:0/2532291639 conn(0x7fc68c0ffe70 msgr2=0x7fc68c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 -- 172.21.15.99:0/2532291639 shutdown_connections 2024-03-23T23:41:28.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.289+0000 7fc693e00700 1 -- 172.21.15.99:0/2532291639 wait complete. 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 Processor -- start 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 -- start start 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 0x7fc68c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:28.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c1146d0 con 0x7fc68c1045e0 2024-03-23T23:41:28.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c114550 con 0x7fc68c104f90 2024-03-23T23:41:28.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.291+0000 7fc693e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c114850 con 0x7fc68c1096c0 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42348/0 (socket says 172.21.15.99:42348) 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 -- 172.21.15.99:0/2808239287 learned_addr learned my addr 172.21.15.99:0/2808239287 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc691e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc691400700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 0x7fc68c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 -- 172.21.15.99:0/2808239287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 msgr2=0x7fc68c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 -- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 msgr2=0x7fc68c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 0x7fc68c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 -- 172.21.15.99:0/2808239287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c19b090 con 0x7fc68c104f90 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc691e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.292+0000 7fc690a00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b2e0 tx=0x7fc67c00b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:28.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.293+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c026070 con 0x7fc68c104f90 2024-03-23T23:41:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.293+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc67c021410 con 0x7fc68c104f90 2024-03-23T23:41:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.293+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c19b320 con 0x7fc68c104f90 2024-03-23T23:41:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.293+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1a7640 con 0x7fc68c104f90 2024-03-23T23:41:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.293+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c017650 con 0x7fc68c104f90 2024-03-23T23:41:28.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.294+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc670005330 con 0x7fc68c104f90 2024-03-23T23:41:28.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.296+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fc67c029070 con 0x7fc68c104f90 2024-03-23T23:41:28.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.298+0000 7fc68a000700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 0x7fc6740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:28.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.299+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc67c0cf830 con 0x7fc68c104f90 2024-03-23T23:41:28.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.299+0000 7fc691400700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 0x7fc6740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:28.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.299+0000 7fc691400700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 0x7fc6740a4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc68c19ba70 tx=0x7fc680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:28.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.300+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc67c013070 con 0x7fc68c104f90 2024-03-23T23:41:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.586+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc6700058f0 con 0x7fc68c104f90 2024-03-23T23:41:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.591+0000 7fc68a000700 1 -- 172.21.15.99:0/2808239287 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262657 (secure 0 0 0) 0x7fc67c09c940 con 0x7fc68c104f90 2024-03-23T23:41:28.596 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 msgr2=0x7fc6740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 0x7fc6740a4e60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc68c19ba70 tx=0x7fc680009260 comp rx=0 tx=0).stop 2024-03-23T23:41:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 msgr2=0x7fc68c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b2e0 tx=0x7fc67c00b6c0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 shutdown_connections 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6740a29a0 0x7fc6740a4e60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c1096c0 0x7fc68c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc68c1045e0 0x7fc68c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 --2- 172.21.15.99:0/2808239287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c104f90 0x7fc68c1a0cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.597+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 >> 172.21.15.99:0/2808239287 conn(0x7fc68c0ffe70 msgr2=0x7fc68c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.598+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 shutdown_connections 2024-03-23T23:41:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:28.598+0000 7fc693e00700 1 -- 172.21.15.99:0/2808239287 wait complete. 2024-03-23T23:41:28.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2808239287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T23:41:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2808239287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T23:41:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2808239287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T23:41:29.529 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":3957482815},{"type":"v1","addr":"172.21.15.99:6801","nonce":3957482815}]},"active_addr":"172.21.15.99:6801/3957482815","active_change":"2024-03-23T23:33:06.585700+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.99:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":559950094}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":769837313}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":475807309}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":348256676}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3766257917}]}]} 2024-03-23T23:41:29.535 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-23T23:41:29.535 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-23T23:41:29.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T23:41:29.966 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:29 smithi099 ceph-mon[31715]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-23T23:41:30.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:29 smithi149 ceph-mon[36083]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-23T23:41:30.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:29 smithi174 ceph-mon[37002]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-23T23:41:31.975 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:31 smithi099 ceph-mon[31715]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-23T23:41:32.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:31 smithi149 ceph-mon[36083]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-23T23:41:32.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 -- 172.21.15.99:0/2852712265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 msgr2=0x7f03901026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:32.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 --2- 172.21.15.99:0/2852712265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f03901026a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f03800099c0 tx=0x7f0380009cd0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 -- 172.21.15.99:0/2852712265 shutdown_connections 2024-03-23T23:41:32.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 --2- 172.21.15.99:0/2852712265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 0x7f039010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 --2- 172.21.15.99:0/2852712265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f03901026a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 --2- 172.21.15.99:0/2852712265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 0x7f0390106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 -- 172.21.15.99:0/2852712265 >> 172.21.15.99:0/2852712265 conn(0x7f03900fdbb0 msgr2=0x7f03900fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 -- 172.21.15.99:0/2852712265 shutdown_connections 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.147+0000 7f0395400700 1 -- 172.21.15.99:0/2852712265 wait complete. 2024-03-23T23:41:32.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.149+0000 7f0395400700 1 Processor -- start 2024-03-23T23:41:32.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.149+0000 7f0395400700 1 -- start start 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 0x7f03900749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 0x7f0390074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03901123d0 con 0x7f0390102c70 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390112250 con 0x7f03901073a0 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0395400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390112550 con 0x7f03901022c0 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:51006/0 (socket says 172.21.15.99:51006) 2024-03-23T23:41:32.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 -- 172.21.15.99:0/2271574163 learned_addr learned my addr 172.21.15.99:0/2271574163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:32.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f0394a00700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 0x7f0390074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:32.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038f400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 0x7f03900749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:32.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 -- 172.21.15.99:0/2271574163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 msgr2=0x7f0390074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:32.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 0x7f0390074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 -- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 msgr2=0x7f03900749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 0x7f03900749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.150+0000 7f038fe00700 1 -- 172.21.15.99:0/2271574163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03901a6d60 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f038fe00700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0380005910 tx=0x7f038000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380027070 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0380003680 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03800173f0 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03901a6ff0 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.151+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03901a73f0 con 0x7f03901022c0 2024-03-23T23:41:32.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.152+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0390068e50 con 0x7f03901022c0 2024-03-23T23:41:32.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.154+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f038002a070 con 0x7f03901022c0 2024-03-23T23:41:32.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.156+0000 7f038cc00700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 0x7f03740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:32.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.156+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03800cfaf0 con 0x7f03901022c0 2024-03-23T23:41:32.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.156+0000 7f038f400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 0x7f03740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:32.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.157+0000 7f038f400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 0x7f03740a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f037c005950 tx=0x7f037c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:32.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.161+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0380013070 con 0x7f03901022c0 2024-03-23T23:41:32.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:31 smithi174 ceph-mon[37002]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 16 op/s 2024-03-23T23:41:32.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.443+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f039002dbf0 con 0x7f03901022c0 2024-03-23T23:41:32.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.446+0000 7f038cc00700 1 -- 172.21.15.99:0/2271574163 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f038009cc90 con 0x7f03901022c0 2024-03-23T23:41:32.446 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:32.446 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:41:11.233681+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:32.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.449+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 msgr2=0x7f03740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:32.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.449+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 0x7f03740a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f037c005950 tx=0x7f037c0058e0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.449+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 msgr2=0x7f0390074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.449+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0380005910 tx=0x7f038000fa00 comp rx=0 tx=0).stop 2024-03-23T23:41:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 shutdown_connections 2024-03-23T23:41:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f03740a29a0 0x7f03740a4e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03901073a0 0x7f0390074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03901022c0 0x7f0390074490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 --2- 172.21.15.99:0/2271574163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0390102c70 0x7f03900749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 >> 172.21.15.99:0/2271574163 conn(0x7f03900fdbb0 msgr2=0x7f03900fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 shutdown_connections 2024-03-23T23:41:32.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:32.450+0000 7f0395400700 1 -- 172.21.15.99:0/2271574163 wait complete. 2024-03-23T23:41:33.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:32 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2271574163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:33.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:32 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2271574163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:33.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:32 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2271574163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:33.337 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-23T23:41:33.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T23:41:33.781 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:34.045 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:33 smithi149 ceph-mon[36083]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-23T23:41:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:33 smithi099 ceph-mon[31715]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-23T23:41:34.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:33 smithi174 ceph-mon[37002]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-23T23:41:35.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.992+0000 7f223a000700 1 -- 172.21.15.99:0/781623724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 msgr2=0x7f2234108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:35.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.992+0000 7f223a000700 1 --2- 172.21.15.99:0/781623724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f2234108fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2220009920 tx=0x7f2220009c30 comp rx=0 tx=0).stop 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 -- 172.21.15.99:0/781623724 shutdown_connections 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 --2- 172.21.15.99:0/781623724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 0x7f2234111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 --2- 172.21.15.99:0/781623724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 0x7f22341049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 --2- 172.21.15.99:0/781623724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f2234108fe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 -- 172.21.15.99:0/781623724 >> 172.21.15.99:0/781623724 conn(0x7f22340ffe90 msgr2=0x7f22341022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 -- 172.21.15.99:0/781623724 shutdown_connections 2024-03-23T23:41:35.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.993+0000 7f223a000700 1 -- 172.21.15.99:0/781623724 wait complete. 2024-03-23T23:41:35.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:35 smithi099 ceph-mon[31715]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T23:41:35.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.994+0000 7f223a000700 1 Processor -- start 2024-03-23T23:41:35.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.994+0000 7f223a000700 1 -- start start 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 0x7f22341a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 0x7f223419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341146d0 con 0x7f22341045e0 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234114550 con 0x7f22341096c0 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f223a000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234114850 con 0x7f2234104f90 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f2232a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f2233400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 0x7f22341a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f2232a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:51020/0 (socket says 172.21.15.99:51020) 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f2232a00700 1 -- 172.21.15.99:0/637998252 learned_addr learned my addr 172.21.15.99:0/637998252 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.995+0000 7f2233e00700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 0x7f223419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 -- 172.21.15.99:0/637998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 msgr2=0x7f223419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 0x7f223419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:35.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 -- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 msgr2=0x7f22341a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:35.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 0x7f22341a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:35.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 -- 172.21.15.99:0/637998252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223419b1d0 con 0x7f2234104f90 2024-03-23T23:41:35.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f2232a00700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f222000a010 tx=0x7f222000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:35.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220005140 con 0x7f2234104f90 2024-03-23T23:41:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f222000b560 con 0x7f2234104f90 2024-03-23T23:41:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223419b460 con 0x7f2234104f90 2024-03-23T23:41:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200039e0 con 0x7f2234104f90 2024-03-23T23:41:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.996+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a7780 con 0x7f2234104f90 2024-03-23T23:41:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.998+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2234105ac0 con 0x7f2234104f90 2024-03-23T23:41:35.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:35.999+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f222000baa0 con 0x7f2234104f90 2024-03-23T23:41:35.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.001+0000 7f222be00700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 0x7f22180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:36.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.001+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f22200cfa50 con 0x7f2234104f90 2024-03-23T23:41:36.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.001+0000 7f2233400700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 0x7f22180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:36.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.002+0000 7f2233400700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 0x7f22180a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f223419bbb0 tx=0x7f2224009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:36.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.009+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2220013030 con 0x7f2234104f90 2024-03-23T23:41:36.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:35 smithi149 ceph-mon[36083]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T23:41:36.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:35 smithi174 ceph-mon[37002]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T23:41:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.241+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223402da30 con 0x7f2234104f90 2024-03-23T23:41:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.243+0000 7f222be00700 1 -- 172.21.15.99:0/637998252 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f222009cad0 con 0x7f2234104f90 2024-03-23T23:41:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:41:36.243 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:41:11.233681+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.246+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 msgr2=0x7f22180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.246+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 0x7f22180a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f223419bbb0 tx=0x7f2224009310 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.246+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 msgr2=0x7f22341a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f222000a010 tx=0x7f222000fa70 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 shutdown_connections 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f22180a2920 0x7f22180a4de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22341096c0 0x7f223419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22341045e0 0x7f22341a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 --2- 172.21.15.99:0/637998252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2234104f90 0x7f22341a0df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 >> 172.21.15.99:0/637998252 conn(0x7f22340ffe90 msgr2=0x7f2234102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 shutdown_connections 2024-03-23T23:41:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:36.247+0000 7f223a000700 1 -- 172.21.15.99:0/637998252 wait complete. 2024-03-23T23:41:36.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-23T23:41:36.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-23T23:41:36.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-23T23:41:36.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-23T23:41:36.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-23T23:41:36.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-23T23:41:36.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-23T23:41:36.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-23T23:41:36.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-23T23:41:36.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-23T23:41:36.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-23T23:41:36.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-23T23:41:37.116 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:36 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/637998252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:37.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:36 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/637998252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:37.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:36 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/637998252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:41:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:37 smithi099 ceph-mon[31715]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-23T23:41:38.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:37 smithi149 ceph-mon[36083]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-23T23:41:38.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:37 smithi174 ceph-mon[37002]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-23T23:41:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.555 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.559 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.560 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.583 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.585 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.606 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.618 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.620 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:41:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:39 smithi099 ceph-mon[31715]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:41:40.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:39 smithi149 ceph-mon[36083]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:41:40.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:39 smithi174 ceph-mon[37002]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:41:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:41 smithi099 ceph-mon[31715]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T23:41:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:41 smithi149 ceph-mon[36083]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T23:41:42.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:41 smithi174 ceph-mon[37002]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-23T23:41:43.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:43 smithi099 ceph-mon[31715]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T23:41:44.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:43 smithi149 ceph-mon[36083]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T23:41:44.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:43 smithi174 ceph-mon[37002]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-23T23:41:45.983 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:45 smithi099 ceph-mon[31715]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:41:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:45 smithi149 ceph-mon[36083]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:41:46.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:45 smithi174 ceph-mon[37002]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 -- 172.21.15.99:0/69238219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f41017f0 msgr2=0x7f78f410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 --2- 172.21.15.99:0/69238219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f41017f0 0x7f78f410e4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f78e00098f0 tx=0x7f78e0009c00 comp rx=0 tx=0).stop 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 -- 172.21.15.99:0/69238219 shutdown_connections 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 --2- 172.21.15.99:0/69238219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f410e9f0 0x7f78f4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 --2- 172.21.15.99:0/69238219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f4100e40 0x7f78f4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 --2- 172.21.15.99:0/69238219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f41017f0 0x7f78f410e4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 -- 172.21.15.99:0/69238219 >> 172.21.15.99:0/69238219 conn(0x7f78f40fca60 msgr2=0x7f78f40fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 -- 172.21.15.99:0/69238219 shutdown_connections 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.558+0000 7f78f3200700 1 -- 172.21.15.99:0/69238219 wait complete. 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.559+0000 7f78f3200700 1 Processor -- start 2024-03-23T23:41:46.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 -- start start 2024-03-23T23:41:46.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f4100e40 0x7f78f41a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:46.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 0x7f78f41a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:46.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f410e9f0 0x7f78f419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4114630 con 0x7f78f410e9f0 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41144b0 con 0x7f78f4100e40 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.560+0000 7f78f3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41147b0 con 0x7f78f41017f0 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 0x7f78f41a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 0x7f78f41a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40904/0 (socket says 172.21.15.99:40904) 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 -- 172.21.15.99:0/2094349267 learned_addr learned my addr 172.21.15.99:0/2094349267 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1e00700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f4100e40 0x7f78f41a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f2800700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f410e9f0 0x7f78f419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 -- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f4100e40 msgr2=0x7f78f41a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78f4100e40 0x7f78f41a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 -- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f410e9f0 msgr2=0x7f78f419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f410e9f0 0x7f78f419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f419b110 con 0x7f78f41017f0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.561+0000 7f78f1400700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 0x7f78f41a0ce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005950 tx=0x7f78e000fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.562+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00051f0 con 0x7f78f41017f0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.562+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78e000b560 con 0x7f78f41017f0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.562+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f419b3a0 con 0x7f78f41017f0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.562+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003ba0 con 0x7f78f41017f0 2024-03-23T23:41:46.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.562+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f41a7670 con 0x7f78f41017f0 2024-03-23T23:41:46.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.563+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78f4062920 con 0x7f78f41017f0 2024-03-23T23:41:46.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.565+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f78e000ba80 con 0x7f78f41017f0 2024-03-23T23:41:46.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78eaa00700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f78d80a29a0 0x7f78d80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78e00cfa60 con 0x7f78f41017f0 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78f1e00700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f78d80a29a0 0x7f78d80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78eaa00700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] conn(0x7f78d80ac180 0x7f78d80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78e0001d30 con 0x7f78d80ac180 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.566+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78e00d42a0 con 0x7f78f41017f0 2024-03-23T23:41:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.567+0000 7f78f2800700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] conn(0x7f78d80ac180 0x7f78d80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.567+0000 7f78f1e00700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f78d80a29a0 0x7f78d80a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f78f4111240 tx=0x7f78e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.567+0000 7f78f2800700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] conn(0x7f78d80ac180 0x7f78d80ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.568+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== osd.7 v2:172.21.15.149:6816/1916985722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f78e0001d30 con 0x7f78d80ac180 2024-03-23T23:41:46.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.596+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 --> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78f4062da0 con 0x7f78d80ac180 2024-03-23T23:41:46.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.597+0000 7f78eaa00700 1 -- 172.21.15.99:0/2094349267 <== osd.7 v2:172.21.15.149:6816/1916985722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f78f4062da0 con 0x7f78d80ac180 2024-03-23T23:41:46.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.597+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] conn(0x7f78d80ac180 msgr2=0x7f78d80ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.597+0000 7f78f3200700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] conn(0x7f78d80ac180 0x7f78d80ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:46.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.597+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f78d80a29a0 msgr2=0x7f78d80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.597+0000 7f78f3200700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f78d80a29a0 0x7f78d80a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f78f4111240 tx=0x7f78e4009260 comp rx=0 tx=0).stop 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 msgr2=0x7f78f41a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 --2- 172.21.15.99:0/2094349267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78f41017f0 0x7f78f41a0ce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005950 tx=0x7f78e000fb20 comp rx=0 tx=0).stop 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f2800700 1 -- 172.21.15.99:0/2094349267 reap_dead start 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 shutdown_connections 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 >> 172.21.15.99:0/2094349267 conn(0x7f78f40fca60 msgr2=0x7f78f40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 shutdown_connections 2024-03-23T23:41:46.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:46.598+0000 7f78f3200700 1 -- 172.21.15.99:0/2094349267 wait complete. 2024-03-23T23:41:48.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:47 smithi099 ceph-mon[31715]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:41:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:47 smithi149 ceph-mon[36083]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:41:48.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:47 smithi174 ceph-mon[37002]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:41:49.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:49 smithi099 ceph-mon[31715]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:50.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:49 smithi149 ceph-mon[36083]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:50.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:49 smithi174 ceph-mon[37002]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:50.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 -- 172.21.15.99:0/1627706508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e68102c80 msgr2=0x7f6e68106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/1627706508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e68102c80 0x7f6e68106cd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e540099c0 tx=0x7f6e54009cd0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 -- 172.21.15.99:0/1627706508 shutdown_connections 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/1627706508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681073b0 0x7f6e6810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/1627706508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e681022d0 0x7f6e681026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/1627706508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e68102c80 0x7f6e68106cd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 -- 172.21.15.99:0/1627706508 >> 172.21.15.99:0/1627706508 conn(0x7f6e680fdb60 msgr2=0x7f6e680fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 -- 172.21.15.99:0/1627706508 shutdown_connections 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.896+0000 7f6e6ce00700 1 -- 172.21.15.99:0/1627706508 wait complete. 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.898+0000 7f6e6ce00700 1 Processor -- start 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.898+0000 7f6e6ce00700 1 -- start start 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.898+0000 7f6e6ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681022d0 0x7f6e68074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.898+0000 7f6e6ce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e68102c80 0x7f6e680749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.898+0000 7f6e6ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e681073b0 0x7f6e68074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e6ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e681123c0 con 0x7f6e681022d0 2024-03-23T23:41:50.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e6ce00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68112240 con 0x7f6e68102c80 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e6ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68112540 con 0x7f6e681073b0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681022d0 0x7f6e68074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e681073b0 0x7f6e68074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681022d0 0x7f6e68074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42072/0 (socket says 172.21.15.99:42072) 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67400700 1 -- 172.21.15.99:0/4091131991 learned_addr learned my addr 172.21.15.99:0/4091131991 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e66a00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e68102c80 0x7f6e680749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 -- 172.21.15.99:0/4091131991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e68102c80 msgr2=0x7f6e680749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e68102c80 0x7f6e680749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 -- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681022d0 msgr2=0x7f6e68074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e681022d0 0x7f6e68074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.899+0000 7f6e67e00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e681a6dc0 con 0x7f6e681073b0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e67e00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e681073b0 0x7f6e68074f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000efc0 tx=0x7f6e6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:50.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600092e0 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e60004bb0 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e681a7050 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e681a75b0 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.900+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60004280 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.901+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e48001030 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.903+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6e6001b030 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.904+0000 7f6e5fe00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6e4c0a29a0 0x7f6e4c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6e600ce410 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e67400700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6e4c0a29a0 0x7f6e4c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e5fe00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] conn(0x7f6e4c0ac180 0x7f6e4c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e60022e50 con 0x7f6e4c0ac180 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e600d5050 con 0x7f6e681073b0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e66a00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] conn(0x7f6e4c0ac180 0x7f6e4c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.905+0000 7f6e67400700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6e4c0a29a0 0x7f6e4c0a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009d20 tx=0x7f6e58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:50.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.906+0000 7f6e66a00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] conn(0x7f6e4c0ac180 0x7f6e4c0ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.907+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== osd.8 v2:172.21.15.174:6816/2258533279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6e60022e50 con 0x7f6e4c0ac180 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.933+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 --> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e48002e60 con 0x7f6e4c0ac180 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e5fe00700 1 -- 172.21.15.99:0/4091131991 <== osd.8 v2:172.21.15.174:6816/2258533279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6e48002e60 con 0x7f6e4c0ac180 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] conn(0x7f6e4c0ac180 msgr2=0x7f6e4c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] conn(0x7f6e4c0ac180 0x7f6e4c0ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6e4c0a29a0 msgr2=0x7f6e4c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6e4c0a29a0 0x7f6e4c0a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009d20 tx=0x7f6e58009260 comp rx=0 tx=0).stop 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e681073b0 msgr2=0x7f6e68074f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e6ce00700 1 --2- 172.21.15.99:0/4091131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e681073b0 0x7f6e68074f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000efc0 tx=0x7f6e6000c3a0 comp rx=0 tx=0).stop 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.934+0000 7f6e67e00700 1 -- 172.21.15.99:0/4091131991 reap_dead start 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.935+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 shutdown_connections 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.935+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 >> 172.21.15.99:0/4091131991 conn(0x7f6e680fdb60 msgr2=0x7f6e680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.935+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 shutdown_connections 2024-03-23T23:41:50.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:49.935+0000 7f6e6ce00700 1 -- 172.21.15.99:0/4091131991 wait complete. 2024-03-23T23:41:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:51 smithi099 ceph-mon[31715]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:41:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:51 smithi149 ceph-mon[36083]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:41:52.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:51 smithi174 ceph-mon[37002]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:41:52.669 INFO:teuthology.orchestra.run.smithi099.stdout:219043332141 2024-03-23T23:41:52.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-23T23:41:53.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 -- 172.21.15.99:0/3645895986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 msgr2=0x7fc9a4106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 --2- 172.21.15.99:0/3645895986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a4106dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc990009990 tx=0x7fc990009ca0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 -- 172.21.15.99:0/3645895986 shutdown_connections 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 --2- 172.21.15.99:0/3645895986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a41074b0 0x7fc9a410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 --2- 172.21.15.99:0/3645895986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 0x7fc9a41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 --2- 172.21.15.99:0/3645895986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a4106dd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 -- 172.21.15.99:0/3645895986 >> 172.21.15.99:0/3645895986 conn(0x7fc9a40fdc40 msgr2=0x7fc9a4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 -- 172.21.15.99:0/3645895986 shutdown_connections 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.927+0000 7fc9abc00700 1 -- 172.21.15.99:0/3645895986 wait complete. 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.928+0000 7fc9abc00700 1 Processor -- start 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 -- start start 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 0x7fc9a41a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a41a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a41074b0 0x7fc9a419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a41124e0 con 0x7fc9a41074b0 2024-03-23T23:41:53.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4112360 con 0x7fc9a4102d80 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.929+0000 7fc9abc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4112660 con 0x7fc9a41023d0 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a9200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 0x7fc9a41a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a9200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 0x7fc9a41a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57010/0 (socket says 172.21.15.99:57010) 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a9c00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a41074b0 0x7fc9a419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a41a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a9200700 1 -- 172.21.15.99:0/4036087731 learned_addr learned my addr 172.21.15.99:0/4036087731 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 -- 172.21.15.99:0/4036087731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 msgr2=0x7fc9a41a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9a41023d0 0x7fc9a41a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 -- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a41074b0 msgr2=0x7fc9a419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9a41074b0 0x7fc9a419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.930+0000 7fc9a3e00700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a419b140 con 0x7fc9a4102d80 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9a3e00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a41a0d60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9900098e0 tx=0x7fc9900057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc990027070 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc99000be90 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9900173f0 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a419b3d0 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.931+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a41a76f0 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.932+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9a40ffa90 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.936+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fc99002a070 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.937+0000 7fc9a1600700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc98c0a29a0 0x7fc98c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.937+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9900cf980 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.937+0000 7fc9a9200700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc98c0a29a0 0x7fc98c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.938+0000 7fc9a1600700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] conn(0x7fc98c0ac050 0x7fc98c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.938+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc990001d30 con 0x7fc98c0ac050 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.938+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9900d42a0 con 0x7fc9a4102d80 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.938+0000 7fc9a9c00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] conn(0x7fc98c0ac050 0x7fc98c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.938+0000 7fc9a9200700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc98c0a29a0 0x7fc98c0a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9a419bb20 tx=0x7fc998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.939+0000 7fc9a9c00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] conn(0x7fc98c0ac050 0x7fc98c0ae470 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.940+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== osd.0 v2:172.21.15.99:6802/1797931451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc990001d30 con 0x7fc98c0ac050 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.961+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 --> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc9a41038b0 con 0x7fc98c0ac050 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.961+0000 7fc9a1600700 1 -- 172.21.15.99:0/4036087731 <== osd.0 v2:172.21.15.99:6802/1797931451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc9a41038b0 con 0x7fc98c0ac050 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] conn(0x7fc98c0ac050 msgr2=0x7fc98c0ae470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] conn(0x7fc98c0ac050 0x7fc98c0ae470 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc98c0a29a0 msgr2=0x7fc98c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc98c0a29a0 0x7fc98c0a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9a419bb20 tx=0x7fc998009260 comp rx=0 tx=0).stop 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 msgr2=0x7fc9a41a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 --2- 172.21.15.99:0/4036087731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9a4102d80 0x7fc9a41a0d60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9900098e0 tx=0x7fc9900057c0 comp rx=0 tx=0).stop 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9a9c00700 1 -- 172.21.15.99:0/4036087731 reap_dead start 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 shutdown_connections 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 >> 172.21.15.99:0/4036087731 conn(0x7fc9a40fdc40 msgr2=0x7fc9a40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 shutdown_connections 2024-03-23T23:41:53.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:52.962+0000 7fc9abc00700 1 -- 172.21.15.99:0/4036087731 wait complete. 2024-03-23T23:41:54.097 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:53 smithi099 ceph-mon[31715]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:41:54.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:53 smithi149 ceph-mon[36083]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:41:54.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:53 smithi174 ceph-mon[37002]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:41:54.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 -- 172.21.15.99:0/3965880595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103240 msgr2=0x7f8954103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 --2- 172.21.15.99:0/3965880595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103240 0x7f8954103620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89440098f0 tx=0x7f8944009c00 comp rx=0 tx=0).stop 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 -- 172.21.15.99:0/3965880595 shutdown_connections 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 --2- 172.21.15.99:0/3965880595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f895410c670 0x7f895410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 --2- 172.21.15.99:0/3965880595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103240 0x7f8954103620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 --2- 172.21.15.99:0/3965880595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103b60 0x7f895410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.257+0000 7f8953e00700 1 -- 172.21.15.99:0/3965880595 >> 172.21.15.99:0/3965880595 conn(0x7f89540fca70 msgr2=0x7f89540fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.258+0000 7f8953e00700 1 -- 172.21.15.99:0/3965880595 shutdown_connections 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.259+0000 7f8953e00700 1 -- 172.21.15.99:0/3965880595 wait complete. 2024-03-23T23:41:54.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.260+0000 7f8953e00700 1 Processor -- start 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 -- start start 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103240 0x7f89541a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103b60 0x7f89541a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f895410c670 0x7f895419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954112400 con 0x7f895410c670 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954112280 con 0x7f8954103b60 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8953e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954112580 con 0x7f8954103240 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8952a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103240 0x7f89541a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8952000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103b60 0x7f89541a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8952a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103240 0x7f89541a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57036/0 (socket says 172.21.15.99:57036) 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.261+0000 7f8952a00700 1 -- 172.21.15.99:0/1513420248 learned_addr learned my addr 172.21.15.99:0/1513420248 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8953400700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f895410c670 0x7f895419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 -- 172.21.15.99:0/1513420248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103240 msgr2=0x7f89541a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8954103240 0x7f89541a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 -- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f895410c670 msgr2=0x7f895419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f895410c670 0x7f895419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895419b080 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8952000700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103b60 0x7f89541a0ca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f894000ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894000ca30 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.262+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895419b370 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.263+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8940004500 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.263+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940010430 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.263+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a7630 con 0x7f8954103b60 2024-03-23T23:41:54.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.264+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89541029a0 con 0x7f8954103b60 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.266+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8940003f60 con 0x7f8954103b60 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.267+0000 7f894b400700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89380a29a0 0x7f89380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.267+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89400cecc0 con 0x7f8954103b60 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.267+0000 7f8952a00700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89380a29a0 0x7f89380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.268+0000 7f894b400700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] conn(0x7f89380ac180 0x7f89380ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.268+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f894001ee50 con 0x7f89380ac180 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.268+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89400d12e0 con 0x7f8954103b60 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.268+0000 7f8953400700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] conn(0x7f89380ac180 0x7f89380ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.268+0000 7f8952a00700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89380a29a0 0x7f89380a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8944009fd0 tx=0x7f8944019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:54.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.269+0000 7f8953400700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] conn(0x7f89380ac180 0x7f89380ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.270+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== osd.9 v2:172.21.15.99:6826/4103401767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f894001ee50 con 0x7f89380ac180 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.299+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 --> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f895406afb0 con 0x7f89380ac180 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.301+0000 7f894b400700 1 -- 172.21.15.99:0/1513420248 <== osd.9 v2:172.21.15.99:6826/4103401767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f895406afb0 con 0x7f89380ac180 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.302+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] conn(0x7f89380ac180 msgr2=0x7f89380ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.302+0000 7f8953e00700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] conn(0x7f89380ac180 0x7f89380ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.306+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89380a29a0 msgr2=0x7f89380a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.306+0000 7f8953e00700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89380a29a0 0x7f89380a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8944009fd0 tx=0x7f8944019040 comp rx=0 tx=0).stop 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.306+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103b60 msgr2=0x7f89541a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.306+0000 7f8953e00700 1 --2- 172.21.15.99:0/1513420248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954103b60 0x7f89541a0ca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f894000ede0 comp rx=0 tx=0).stop 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.308+0000 7f8953400700 1 -- 172.21.15.99:0/1513420248 reap_dead start 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.308+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 shutdown_connections 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.308+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 >> 172.21.15.99:0/1513420248 conn(0x7f89540fca70 msgr2=0x7f89540fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.309+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 shutdown_connections 2024-03-23T23:41:54.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:54.309+0000 7f8953e00700 1 -- 172.21.15.99:0/1513420248 wait complete. 2024-03-23T23:41:55.257 INFO:teuthology.orchestra.run.smithi099.stdout:34359738453 2024-03-23T23:41:55.257 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-23T23:41:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:55 smithi099 ceph-mon[31715]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:55 smithi149 ceph-mon[36083]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:56.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:55 smithi174 ceph-mon[37002]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:56.233 INFO:teuthology.orchestra.run.smithi099.stdout:253403070503 2024-03-23T23:41:56.234 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-23T23:41:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 -- 172.21.15.99:0/1327977021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 msgr2=0x7f5644108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 --2- 172.21.15.99:0/1327977021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f5644108fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56300099c0 tx=0x7f5630009cd0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 -- 172.21.15.99:0/1327977021 shutdown_connections 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 --2- 172.21.15.99:0/1327977021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56441096b0 0x7f5644111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 --2- 172.21.15.99:0/1327977021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 0x7f56441049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 --2- 172.21.15.99:0/1327977021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f5644108fd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.486+0000 7f5649c00700 1 -- 172.21.15.99:0/1327977021 >> 172.21.15.99:0/1327977021 conn(0x7f56440ffea0 msgr2=0x7f56441022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.487+0000 7f5649c00700 1 -- 172.21.15.99:0/1327977021 shutdown_connections 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.487+0000 7f5649c00700 1 -- 172.21.15.99:0/1327977021 wait complete. 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.488+0000 7f5649c00700 1 Processor -- start 2024-03-23T23:41:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.488+0000 7f5649c00700 1 -- start start 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.488+0000 7f5649c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 0x7f56441a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.488+0000 7f5649c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f56441a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5649c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56441096b0 0x7f564419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5649c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441146c0 con 0x7f56441045d0 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5649c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5644114540 con 0x7f56441096b0 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5649c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5644114840 con 0x7f5644104f80 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f56441a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f56441a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57058/0 (socket says 172.21.15.99:57058) 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 -- 172.21.15.99:0/4218887580 learned_addr learned my addr 172.21.15.99:0/4218887580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642a00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 0x7f56441a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5643400700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56441096b0 0x7f564419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 -- 172.21.15.99:0/4218887580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56441096b0 msgr2=0x7f564419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56441096b0 0x7f564419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 -- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 msgr2=0x7f56441a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 0x7f56441a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642000700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564419b100 con 0x7f5644104f80 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.489+0000 7f5642a00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56441045d0 0x7f56441a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f5642000700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f56441a0d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f563000f690 tx=0x7f5630005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630027070 con 0x7f5644104f80 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5630003680 con 0x7f5644104f80 2024-03-23T23:41:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300173f0 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564419b390 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.490+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56441a76b0 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.491+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5644062920 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.494+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f563002a070 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.496+0000 7f563b400700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f56280a29a0 0x7f56280a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.496+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56300cfe00 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.496+0000 7f5642a00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f56280a29a0 0x7f56280a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.497+0000 7f563b400700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] conn(0x7f56280ac180 0x7f56280ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.497+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56280aec50 con 0x7f56280ac180 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.497+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56300d01b0 con 0x7f5644104f80 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.497+0000 7f5643400700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] conn(0x7f56280ac180 0x7f56280ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.498+0000 7f5642a00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f56280a29a0 0x7f56280a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f564419bae0 tx=0x7f5634009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.498+0000 7f5643400700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] conn(0x7f56280ac180 0x7f56280ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.499+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== osd.3 v2:172.21.15.99:6810/1963358194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f56280aec50 con 0x7f56280ac180 2024-03-23T23:41:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.531+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 --> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5644062da0 con 0x7f56280ac180 2024-03-23T23:41:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.531+0000 7f563b400700 1 -- 172.21.15.99:0/4218887580 <== osd.3 v2:172.21.15.99:6810/1963358194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5644062da0 con 0x7f56280ac180 2024-03-23T23:41:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] conn(0x7f56280ac180 msgr2=0x7f56280ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] conn(0x7f56280ac180 0x7f56280ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f56280a29a0 msgr2=0x7f56280a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f56280a29a0 0x7f56280a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f564419bae0 tx=0x7f5634009260 comp rx=0 tx=0).stop 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 msgr2=0x7f56441a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5649c00700 1 --2- 172.21.15.99:0/4218887580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5644104f80 0x7f56441a0d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f563000f690 tx=0x7f5630005f10 comp rx=0 tx=0).stop 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.532+0000 7f5643400700 1 -- 172.21.15.99:0/4218887580 reap_dead start 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.533+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 shutdown_connections 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.533+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 >> 172.21.15.99:0/4218887580 conn(0x7f56440ffea0 msgr2=0x7f5644102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.533+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 shutdown_connections 2024-03-23T23:41:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:55.533+0000 7f5649c00700 1 -- 172.21.15.99:0/4218887580 wait complete. 2024-03-23T23:41:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:57 smithi099 ceph-mon[31715]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:57 smithi149 ceph-mon[36083]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:58.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:57 smithi174 ceph-mon[37002]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:41:58.267 INFO:teuthology.orchestra.run.smithi099.stdout:107374182467 2024-03-23T23:41:58.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 -- 172.21.15.99:0/1461636394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 msgr2=0x7f752c114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 --2- 172.21.15.99:0/1461636394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 0x7f752c114da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7518009990 tx=0x7f7518009ca0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 -- 172.21.15.99:0/1461636394 shutdown_connections 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 --2- 172.21.15.99:0/1461636394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c1152e0 0x7f752c1176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 --2- 172.21.15.99:0/1461636394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c109190 0x7f752c109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 --2- 172.21.15.99:0/1461636394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 0x7f752c114da0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.513+0000 7f7533a00700 1 -- 172.21.15.99:0/1461636394 >> 172.21.15.99:0/1461636394 conn(0x7f752c06d2d0 msgr2=0x7f752c06d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:58.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.514+0000 7f7533a00700 1 -- 172.21.15.99:0/1461636394 shutdown_connections 2024-03-23T23:41:58.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.515+0000 7f7533a00700 1 -- 172.21.15.99:0/1461636394 wait complete. 2024-03-23T23:41:58.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.517+0000 7f7533a00700 1 Processor -- start 2024-03-23T23:41:58.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.517+0000 7f7533a00700 1 -- start start 2024-03-23T23:41:58.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.517+0000 7f7533a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 0x7f752c1a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7533a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 0x7f752c1a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7533a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c1152e0 0x7f752c1a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7533a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c11ad10 con 0x7f752c1152e0 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7533a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c11ab90 con 0x7f752c109ab0 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7533a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c11ae90 con 0x7f752c109190 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7531000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 0x7f752c1a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7531000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 0x7f752c1a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57098/0 (socket says 172.21.15.99:57098) 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7531000700 1 -- 172.21.15.99:0/3955411532 learned_addr learned my addr 172.21.15.99:0/3955411532 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f7531a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c1152e0 0x7f752c1a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.518+0000 7f752be00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 0x7f752c1a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 -- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 msgr2=0x7f752c1a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f752c109ab0 0x7f752c1a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 -- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c1152e0 msgr2=0x7f752c1a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c1152e0 0x7f752c1a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c1a83a0 con 0x7f752c109190 2024-03-23T23:41:58.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f752c1152e0 0x7f752c1a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7531000700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 0x7f752c1a3390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f752000d980 tx=0x7f752000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.519+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520016070 con 0x7f752c109190 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.520+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7520004500 con 0x7f752c109190 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.520+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520004d00 con 0x7f752c109190 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.520+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c1a8690 con 0x7f752c109190 2024-03-23T23:41:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.520+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1aff10 con 0x7f752c109190 2024-03-23T23:41:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.521+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f752c0716e0 con 0x7f752c109190 2024-03-23T23:41:58.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.524+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f752000bbf0 con 0x7f752c109190 2024-03-23T23:41:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.527+0000 7f7529600700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f75140a29a0 0x7f75140a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.527+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f75200cefb0 con 0x7f752c109190 2024-03-23T23:41:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.527+0000 7f7529600700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] conn(0x7f75140ac180 0x7f75140ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.527+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7520015ec0 con 0x7f75140ac180 2024-03-23T23:41:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.527+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75200cf360 con 0x7f752c109190 2024-03-23T23:41:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.528+0000 7f752be00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f75140a29a0 0x7f75140a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.528+0000 7f7531a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] conn(0x7f75140ac180 0x7f75140ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.528+0000 7f7531a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] conn(0x7f75140ac180 0x7f75140ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.529+0000 7f752be00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f75140a29a0 0x7f75140a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f751800b2e0 tx=0x7f751801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.529+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== osd.1 v2:172.21.15.149:6800/809430535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7520015ec0 con 0x7f75140ac180 2024-03-23T23:41:58.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.563+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 --> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f752c06e830 con 0x7f75140ac180 2024-03-23T23:41:58.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.563+0000 7f7529600700 1 -- 172.21.15.99:0/3955411532 <== osd.1 v2:172.21.15.149:6800/809430535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f752c06e830 con 0x7f75140ac180 2024-03-23T23:41:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] conn(0x7f75140ac180 msgr2=0x7f75140ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] conn(0x7f75140ac180 0x7f75140ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f75140a29a0 msgr2=0x7f75140a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f75140a29a0 0x7f75140a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f751800b2e0 tx=0x7f751801a040 comp rx=0 tx=0).stop 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 msgr2=0x7f752c1a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 --2- 172.21.15.99:0/3955411532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c109190 0x7f752c1a3390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f752000d980 tx=0x7f752000dd40 comp rx=0 tx=0).stop 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7531a00700 1 -- 172.21.15.99:0/3955411532 reap_dead start 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 shutdown_connections 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 >> 172.21.15.99:0/3955411532 conn(0x7f752c06d2d0 msgr2=0x7f752c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.564+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 shutdown_connections 2024-03-23T23:41:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:58.565+0000 7f7533a00700 1 -- 172.21.15.99:0/3955411532 wait complete. 2024-03-23T23:41:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 -- 172.21.15.99:0/138935122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 msgr2=0x7fa104106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 --2- 172.21.15.99:0/138935122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa104106e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009990 tx=0x7fa0f0009ca0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 -- 172.21.15.99:0/138935122 shutdown_connections 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 --2- 172.21.15.99:0/138935122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104107510 0x7fa10410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 --2- 172.21.15.99:0/138935122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104102440 0x7fa104102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 --2- 172.21.15.99:0/138935122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa104106e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 -- 172.21.15.99:0/138935122 >> 172.21.15.99:0/138935122 conn(0x7fa1040792f0 msgr2=0x7fa104079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 -- 172.21.15.99:0/138935122 shutdown_connections 2024-03-23T23:41:58.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.318+0000 7fa10a400700 1 -- 172.21.15.99:0/138935122 wait complete. 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.320+0000 7fa10a400700 1 Processor -- start 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.320+0000 7fa10a400700 1 -- start start 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.320+0000 7fa10a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104102440 0x7fa10410e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.320+0000 7fa10a400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa10410ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa10a400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 0x7fa1041a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa10a400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104112540 con 0x7fa104102440 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa10a400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041123c0 con 0x7fa104102df0 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa10a400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041126c0 con 0x7fa104107510 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa103e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 0x7fa1041a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa103400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104102440 0x7fa10410e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa103e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 0x7fa1041a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57082/0 (socket says 172.21.15.99:57082) 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa103e00700 1 -- 172.21.15.99:0/4010234444 learned_addr learned my addr 172.21.15.99:0/4010234444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:41:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa102a00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa10410ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa102a00700 1 -- 172.21.15.99:0/4010234444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 msgr2=0x7fa1041a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.321+0000 7fa102a00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 0x7fa1041a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa102a00700 1 -- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104102440 msgr2=0x7fa10410e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa102a00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104102440 0x7fa10410e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa102a00700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1041a4100 con 0x7fa104102df0 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa103e00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104107510 0x7fa1041a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa103400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa104102440 0x7fa10410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa102a00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa10410ea00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.322+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0026070 con 0x7fa104102df0 2024-03-23T23:41:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.323+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f000b6c0 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.323+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00173f0 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.323+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1041a4390 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.323+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1041a47f0 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.326+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa0f0029070 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.327+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa104100ad0 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.327+0000 7fa0fbe00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa0e80a29a0 0x7fa0e80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.327+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0f00cf860 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa103400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa0e80a29a0 0x7fa0e80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa0fbe00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] conn(0x7fa0e80ac180 0x7fa0e80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0f000fe50 con 0x7fa0e80ac180 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f00d42a0 con 0x7fa104102df0 2024-03-23T23:41:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa103e00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] conn(0x7fa0e80ac180 0x7fa0e80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:41:58.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.328+0000 7fa103400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa0e80a29a0 0x7fa0e80a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009cb0 tx=0x7fa0f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.329+0000 7fa103e00700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] conn(0x7fa0e80ac180 0x7fa0e80ae5a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.330+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== osd.10 v2:172.21.15.149:6824/2851714009 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa0f000fe50 con 0x7fa0e80ac180 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.356+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 --> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa104103910 con 0x7fa0e80ac180 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa0fbe00700 1 -- 172.21.15.99:0/4010234444 <== osd.10 v2:172.21.15.149:6824/2851714009 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa104103910 con 0x7fa0e80ac180 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] conn(0x7fa0e80ac180 msgr2=0x7fa0e80ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] conn(0x7fa0e80ac180 0x7fa0e80ae5a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa0e80a29a0 msgr2=0x7fa0e80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa0e80a29a0 0x7fa0e80a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009cb0 tx=0x7fa0f4009260 comp rx=0 tx=0).stop 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 msgr2=0x7fa10410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 --2- 172.21.15.99:0/4010234444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa104102df0 0x7fa10410ea00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000c00 tx=0x7fa0f00047b0 comp rx=0 tx=0).stop 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa103e00700 1 -- 172.21.15.99:0/4010234444 reap_dead start 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 shutdown_connections 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.357+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 >> 172.21.15.99:0/4010234444 conn(0x7fa1040792f0 msgr2=0x7fa1041000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:41:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.358+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 shutdown_connections 2024-03-23T23:41:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:41:57.358+0000 7fa10a400700 1 -- 172.21.15.99:0/4010234444 wait complete. 2024-03-23T23:41:59.870 INFO:teuthology.orchestra.run.smithi099.stdout:309237645341 2024-03-23T23:41:59.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-23T23:42:00.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:41:59 smithi099 ceph-mon[31715]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-23T23:42:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:41:59 smithi149 ceph-mon[36083]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-23T23:42:00.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:41:59 smithi174 ceph-mon[37002]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-23T23:42:01.676 INFO:teuthology.orchestra.run.smithi099.stdout:287762808865 2024-03-23T23:42:01.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-23T23:42:01.899 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:01 smithi099 ceph-mon[31715]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:02.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:01 smithi149 ceph-mon[36083]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:02.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:01 smithi174 ceph-mon[37002]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:02.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 -- 172.21.15.99:0/887428087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc1074f0 msgr2=0x7fcbfc10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 --2- 172.21.15.99:0/887428087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc1074f0 0x7fcbfc10f0c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400b0d0 tx=0x7fcbf400b3e0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 -- 172.21.15.99:0/887428087 shutdown_connections 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 --2- 172.21.15.99:0/887428087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc1074f0 0x7fcbfc10f0c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 --2- 172.21.15.99:0/887428087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 --2- 172.21.15.99:0/887428087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbfc102dc0 0x7fcbfc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 -- 172.21.15.99:0/887428087 >> 172.21.15.99:0/887428087 conn(0x7fcbfc0fdc60 msgr2=0x7fcbfc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 -- 172.21.15.99:0/887428087 shutdown_connections 2024-03-23T23:42:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.268+0000 7fcc04000700 1 -- 172.21.15.99:0/887428087 wait complete. 2024-03-23T23:42:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.269+0000 7fcc04000700 1 Processor -- start 2024-03-23T23:42:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.269+0000 7fcc04000700 1 -- start start 2024-03-23T23:42:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc102dc0 0x7fcbfc1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbfc1074f0 0x7fcbfc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc112550 con 0x7fcbfc1074f0 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc1123d0 con 0x7fcbfc102410 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc04000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc1126d0 con 0x7fcbfc102dc0 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34266/0 (socket says 172.21.15.99:34266) 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc00c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc102dc0 0x7fcbfc1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 -- 172.21.15.99:0/51716705 learned_addr learned my addr 172.21.15.99:0/51716705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc02000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbfc1074f0 0x7fcbfc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 -- 172.21.15.99:0/51716705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc102dc0 msgr2=0x7fcbfc1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc102dc0 0x7fcbfc1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 -- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbfc1074f0 msgr2=0x7fcbfc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbfc1074f0 0x7fcbfc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc01600700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc19b0e0 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.270+0000 7fcc00c00700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbfc102dc0 0x7fcbfc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcc01600700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1a0770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e9c0 tx=0x7fcbf000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000cbd0 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcbf000cd70 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc19b3d0 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0010430 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.271+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc1a7640 con 0x7fcbfc102410 2024-03-23T23:42:02.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.272+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcbfc0ff000 con 0x7fcbfc102410 2024-03-23T23:42:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.276+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcbf0004750 con 0x7fcbfc102410 2024-03-23T23:42:02.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcbfa000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcbe40a2920 0x7fcbe40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:02.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcbf00cedc0 con 0x7fcbfc102410 2024-03-23T23:42:02.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcc00c00700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcbe40a2920 0x7fcbe40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:02.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcbfa000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] conn(0x7fcbe40ac100 0x7fcbe40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:02.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbe40aebd0 con 0x7fcbe40ac100 2024-03-23T23:42:02.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.277+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf00d13e0 con 0x7fcbfc102410 2024-03-23T23:42:02.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.278+0000 7fcc02000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] conn(0x7fcbe40ac100 0x7fcbe40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:02.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.278+0000 7fcc00c00700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcbe40a2920 0x7fcbe40a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10f570 tx=0x7fcbe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:02.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.278+0000 7fcc02000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] conn(0x7fcbe40ac100 0x7fcbe40ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:02.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.279+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== osd.6 v2:172.21.15.99:6818/1423285526 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcbe40aebd0 con 0x7fcbe40ac100 2024-03-23T23:42:02.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.301+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 --> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcbfc1038f0 con 0x7fcbe40ac100 2024-03-23T23:42:02.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.302+0000 7fcbfa000700 1 -- 172.21.15.99:0/51716705 <== osd.6 v2:172.21.15.99:6818/1423285526 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcbfc1038f0 con 0x7fcbe40ac100 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.302+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] conn(0x7fcbe40ac100 msgr2=0x7fcbe40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.302+0000 7fcc04000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] conn(0x7fcbe40ac100 0x7fcbe40ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcbe40a2920 msgr2=0x7fcbe40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcbe40a2920 0x7fcbe40a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10f570 tx=0x7fcbe8009260 comp rx=0 tx=0).stop 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 msgr2=0x7fcbfc1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 --2- 172.21.15.99:0/51716705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbfc102410 0x7fcbfc1a0770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e9c0 tx=0x7fcbf000ecd0 comp rx=0 tx=0).stop 2024-03-23T23:42:02.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc02000700 1 -- 172.21.15.99:0/51716705 reap_dead start 2024-03-23T23:42:02.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 shutdown_connections 2024-03-23T23:42:02.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 >> 172.21.15.99:0/51716705 conn(0x7fcbfc0fdc60 msgr2=0x7fcbfc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:02.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 shutdown_connections 2024-03-23T23:42:02.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:02.303+0000 7fcc04000700 1 -- 172.21.15.99:0/51716705 wait complete. 2024-03-23T23:42:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2666231978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc102df0 msgr2=0x7f8cbc106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2666231978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc102df0 0x7f8cbc106e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009990 tx=0x7f8ca8009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2666231978 shutdown_connections 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2666231978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cbc107510 0x7f8cbc10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2666231978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 0x7f8cbc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2666231978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc102df0 0x7f8cbc106e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2666231978 >> 172.21.15.99:0/2666231978 conn(0x7f8cbc0792f0 msgr2=0x7f8cbc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2666231978 shutdown_connections 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.045+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2666231978 wait complete. 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.046+0000 7f8cc1e00700 1 Processor -- start 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.046+0000 7f8cc1e00700 1 -- start start 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 0x7f8cbc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cbc102df0 0x7f8cbc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 0x7f8cbc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc112520 con 0x7f8cbc102440 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1123a0 con 0x7f8cbc107510 2024-03-23T23:42:03.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cc1e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1126a0 con 0x7f8cbc102df0 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cbb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 0x7f8cbc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cbb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 0x7f8cbc19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34288/0 (socket says 172.21.15.99:34288) 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.047+0000 7f8cbb400700 1 -- 172.21.15.99:0/2587537915 learned_addr learned my addr 172.21.15.99:0/2587537915 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbaa00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 0x7f8cbc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cba000700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cbc102df0 0x7f8cbc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 -- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cbc102df0 msgr2=0x7f8cbc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cbc102df0 0x7f8cbc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 -- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 msgr2=0x7f8cbc1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 0x7f8cbc1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc19b180 con 0x7f8cbc107510 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbaa00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc102440 0x7f8cbc1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cbb400700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 0x7f8cbc19aa20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c0f0 tx=0x7f8cb400c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.048+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4018020 con 0x7f8cbc107510 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.049+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cb4003a00 con 0x7f8cbc107510 2024-03-23T23:42:03.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.049+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc19b470 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.049+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007500 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.049+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1a7730 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.050+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cbc0756a0 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.053+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8cb4012050 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.054+0000 7f8cc0a00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8ca00a29a0 0x7f8ca00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.054+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8cb40ce7b0 con 0x7f8cbc107510 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cbaa00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8ca00a29a0 0x7f8ca00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cc0a00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] conn(0x7f8ca00ac180 0x7f8ca00ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:03.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb401ee70 con 0x7f8ca00ac180 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cb40ceb60 con 0x7f8cbc107510 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cba000700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] conn(0x7f8ca00ac180 0x7f8ca00ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.055+0000 7f8cbaa00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8ca00a29a0 0x7f8ca00a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc19bb60 tx=0x7f8cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.056+0000 7f8cba000700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] conn(0x7f8ca00ac180 0x7f8ca00ae5a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.056+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== osd.11 v2:172.21.15.174:6824/2854369020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8cb401ee70 con 0x7f8ca00ac180 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.076+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 --> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cbc103910 con 0x7f8ca00ac180 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.076+0000 7f8cc0a00700 1 -- 172.21.15.99:0/2587537915 <== osd.11 v2:172.21.15.174:6824/2854369020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8cbc103910 con 0x7f8ca00ac180 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] conn(0x7f8ca00ac180 msgr2=0x7f8ca00ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] conn(0x7f8ca00ac180 0x7f8ca00ae5a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8ca00a29a0 msgr2=0x7f8ca00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8ca00a29a0 0x7f8ca00a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc19bb60 tx=0x7f8cac009260 comp rx=0 tx=0).stop 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 msgr2=0x7f8cbc19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 --2- 172.21.15.99:0/2587537915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cbc107510 0x7f8cbc19aa20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c0f0 tx=0x7f8cb400c4b0 comp rx=0 tx=0).stop 2024-03-23T23:42:03.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cbb400700 1 -- 172.21.15.99:0/2587537915 reap_dead start 2024-03-23T23:42:03.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 shutdown_connections 2024-03-23T23:42:03.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 >> 172.21.15.99:0/2587537915 conn(0x7f8cbc0792f0 msgr2=0x7f8cbc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:03.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 shutdown_connections 2024-03-23T23:42:03.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.077+0000 7f8cc1e00700 1 -- 172.21.15.99:0/2587537915 wait complete. 2024-03-23T23:42:03.937 INFO:teuthology.orchestra.run.smithi099.stdout:343597383705 2024-03-23T23:42:03.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-23T23:42:04.021 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:03 smithi099 ceph-mon[31715]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:03 smithi149 ceph-mon[36083]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:04.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:03 smithi174 ceph-mon[37002]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:05.233 INFO:teuthology.orchestra.run.smithi099.stdout:197568495669 2024-03-23T23:42:05.234 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 -- 172.21.15.99:0/4175792586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc074ce0 msgr2=0x7f3bcc10e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 --2- 172.21.15.99:0/4175792586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc074ce0 0x7f3bcc10e650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009990 tx=0x7f3bb8009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 -- 172.21.15.99:0/4175792586 shutdown_connections 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 --2- 172.21.15.99:0/4175792586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 0x7f3bcc110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 --2- 172.21.15.99:0/4175792586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 --2- 172.21.15.99:0/4175792586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc074ce0 0x7f3bcc10e650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 -- 172.21.15.99:0/4175792586 >> 172.21.15.99:0/4175792586 conn(0x7f3bcc0792c0 msgr2=0x7f3bcc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 -- 172.21.15.99:0/4175792586 shutdown_connections 2024-03-23T23:42:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.172+0000 7f3bd4000700 1 -- 172.21.15.99:0/4175792586 wait complete. 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.173+0000 7f3bd4000700 1 Processor -- start 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 -- start start 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bcc074ce0 0x7f3bcc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 0x7f3bcc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1146e0 con 0x7f3bcc074ce0 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc114560 con 0x7f3bcc10eb90 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd4000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc114860 con 0x7f3bcc0743c0 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd1600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd1600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc1a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:44492/0 (socket says 172.21.15.99:44492) 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd1600700 1 -- 172.21.15.99:0/112748166 learned_addr learned my addr 172.21.15.99:0/112748166 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd0c00700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bcc074ce0 0x7f3bcc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.174+0000 7f3bd2000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 0x7f3bcc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 -- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 msgr2=0x7f3bcc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 0x7f3bcc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 -- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bcc074ce0 msgr2=0x7f3bcc1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bcc074ce0 0x7f3bcc1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc19b1a0 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd2000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bcc10eb90 0x7f3bcc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd1600700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc1a0880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000e9a0 tx=0x7f3bc000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000ca10 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bc000cbb0 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc19b490 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0018680 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.175+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1a7750 con 0x7f3bcc0743c0 2024-03-23T23:42:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.176+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3bcc062860 con 0x7f3bcc0743c0 2024-03-23T23:42:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.178+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f3bc0047a90 con 0x7f3bcc0743c0 2024-03-23T23:42:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.179+0000 7f3bca000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3bb40a29a0 0x7f3bb40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3bc00ceea0 con 0x7f3bcc0743c0 2024-03-23T23:42:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bd0c00700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3bb40a29a0 0x7f3bb40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bca000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] conn(0x7f3bb40ac180 0x7f3bb40ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bb40aec50 con 0x7f3bb40ac180 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bc00cf250 con 0x7f3bcc0743c0 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.180+0000 7f3bd2000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] conn(0x7f3bb40ac180 0x7f3bb40ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.181+0000 7f3bd0c00700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3bb40a29a0 0x7f3bb40a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc101390 tx=0x7f3bb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.181+0000 7f3bd2000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] conn(0x7f3bb40ac180 0x7f3bb40ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.182+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== osd.4 v2:172.21.15.149:6808/766703080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3bb40aec50 con 0x7f3bb40ac180 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.199+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 --> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3bcc062c90 con 0x7f3bb40ac180 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bca000700 1 -- 172.21.15.99:0/112748166 <== osd.4 v2:172.21.15.149:6808/766703080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3bcc062c90 con 0x7f3bb40ac180 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] conn(0x7f3bb40ac180 msgr2=0x7f3bb40ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] conn(0x7f3bb40ac180 0x7f3bb40ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3bb40a29a0 msgr2=0x7f3bb40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3bb40a29a0 0x7f3bb40a4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc101390 tx=0x7f3bb8005fd0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 msgr2=0x7f3bcc1a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 --2- 172.21.15.99:0/112748166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bcc0743c0 0x7f3bcc1a0880 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000e9a0 tx=0x7f3bc000ecb0 comp rx=0 tx=0).stop 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd2000700 1 -- 172.21.15.99:0/112748166 reap_dead start 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 shutdown_connections 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.200+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 >> 172.21.15.99:0/112748166 conn(0x7f3bcc0792c0 msgr2=0x7f3bcc100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.201+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 shutdown_connections 2024-03-23T23:42:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:03.201+0000 7f3bd4000700 1 -- 172.21.15.99:0/112748166 wait complete. 2024-03-23T23:42:06.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:05 smithi099 ceph-mon[31715]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:06.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:05 smithi149 ceph-mon[36083]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:06.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:05 smithi174 ceph-mon[37002]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:06.614 INFO:teuthology.orchestra.run.smithi099.stdout:141733920832 2024-03-23T23:42:06.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-23T23:42:06.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 -- 172.21.15.99:0/1637431536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04102df0 msgr2=0x7fee04106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 --2- 172.21.15.99:0/1637431536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04102df0 0x7fee04106e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009990 tx=0x7fedf0009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 -- 172.21.15.99:0/1637431536 shutdown_connections 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 --2- 172.21.15.99:0/1637431536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04107510 0x7fee0410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 --2- 172.21.15.99:0/1637431536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee04102440 0x7fee04102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 --2- 172.21.15.99:0/1637431536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04102df0 0x7fee04106e30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 -- 172.21.15.99:0/1637431536 >> 172.21.15.99:0/1637431536 conn(0x7fee040792f0 msgr2=0x7fee04079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 -- 172.21.15.99:0/1637431536 shutdown_connections 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.978+0000 7fee0c200700 1 -- 172.21.15.99:0/1637431536 wait complete. 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.980+0000 7fee0c200700 1 Processor -- start 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.980+0000 7fee0c200700 1 -- start start 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 0x7fee0419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee04102df0 0x7fee0419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04107510 0x7fee041988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04112520 con 0x7fee04102df0 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041123a0 con 0x7fee04107510 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0c200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041126a0 con 0x7fee04102440 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 0x7fee0419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 0x7fee0419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:44520/0 (socket says 172.21.15.99:44520) 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 -- 172.21.15.99:0/3812932223 learned_addr learned my addr 172.21.15.99:0/3812932223 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee08e00700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee04102df0 0x7fee0419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0a200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04107510 0x7fee041988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:06.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 -- 172.21.15.99:0/3812932223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04107510 msgr2=0x7fee041988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04107510 0x7fee041988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 -- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee04102df0 msgr2=0x7fee0419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee04102df0 0x7fee0419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee09800700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee04199040 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.981+0000 7fee0a200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee04107510 0x7fee041988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee09800700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 0x7fee0419e6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ef10 tx=0x7fedf800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800c9c0 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee041992d0 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf8019070 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8007e60 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.982+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee041a55a0 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.983+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee040628a0 con 0x7fee04102440 2024-03-23T23:42:06.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.985+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fedf8012050 con 0x7fee04102440 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.986+0000 7fee02000700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fedec0a2920 0x7fedec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.986+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fedf80cf5b0 con 0x7fee04102440 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee08e00700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fedec0a2920 0x7fedec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee02000700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] conn(0x7fedec0ac100 0x7fedec0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedf800be50 con 0x7fedec0ac100 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf80cf960 con 0x7fee04102440 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee0a200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] conn(0x7fedec0ac100 0x7fedec0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.987+0000 7fee08e00700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fedec0a2920 0x7fedec0a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee04101d40 tx=0x7fedf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.988+0000 7fee0a200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] conn(0x7fedec0ac100 0x7fedec0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:06.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:04.988+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== osd.2 v2:172.21.15.174:6800/3172830280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fedf800be50 con 0x7fedec0ac100 2024-03-23T23:42:06.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.011+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 --> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee04062ca0 con 0x7fedec0ac100 2024-03-23T23:42:06.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee02000700 1 -- 172.21.15.99:0/3812932223 <== osd.2 v2:172.21.15.174:6800/3172830280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee04062ca0 con 0x7fedec0ac100 2024-03-23T23:42:06.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] conn(0x7fedec0ac100 msgr2=0x7fedec0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] conn(0x7fedec0ac100 0x7fedec0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fedec0a2920 msgr2=0x7fedec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fedec0a2920 0x7fedec0a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee04101d40 tx=0x7fedf0005fd0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 msgr2=0x7fee0419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 --2- 172.21.15.99:0/3812932223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04102440 0x7fee0419e6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedf800ef10 tx=0x7fedf800c3a0 comp rx=0 tx=0).stop 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0a200700 1 -- 172.21.15.99:0/3812932223 reap_dead start 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 shutdown_connections 2024-03-23T23:42:06.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 >> 172.21.15.99:0/3812932223 conn(0x7fee040792f0 msgr2=0x7fee041002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:06.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 shutdown_connections 2024-03-23T23:42:06.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:05.012+0000 7fee0c200700 1 -- 172.21.15.99:0/3812932223 wait complete. 2024-03-23T23:42:07.113 INFO:teuthology.orchestra.run.smithi099.stdout:55834574929 2024-03-23T23:42:07.114 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-23T23:42:07.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:06 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-23T23:42:07.168 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:06 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-23T23:42:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-03-23T23:42:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-23T23:42:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-03-23T23:42:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-23T23:42:07.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T23:42:07.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-03-23T23:42:07.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-23T23:42:07.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-23T23:42:07.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-23T23:42:07.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:06 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-23T23:42:07.215 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:07.585 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:07.636 INFO:teuthology.orchestra.run.smithi099.stdout:77309411406 2024-03-23T23:42:07.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-23T23:42:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:07.758 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-23T23:42:08.109 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-23T23:42:08.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:07 smithi099 ceph-mon[31715]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-23T23:42:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T23:42:08.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-23T23:42:08.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-23T23:42:08.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:07 smithi149 ceph-mon[36083]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T23:42:08.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:08.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-03-23T23:42:08.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-23T23:42:08.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-03-23T23:42:08.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-23T23:42:08.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:07 smithi174 ceph-mon[37002]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T23:42:08.914 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:08 smithi099 ceph-mon[31715]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T23:42:09.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:08 smithi149 ceph-mon[36083]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T23:42:09.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:08 smithi174 ceph-mon[37002]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T23:42:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:09 smithi099 ceph-mon[31715]: pgmap v348: 97 pgs: 1 activating+degraded, 1 activating, 2 peering, 93 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 2/306 objects degraded (0.654%); 0 B/s, 0 objects/s recovering 2024-03-23T23:42:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:09 smithi099 ceph-mon[31715]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-23T23:42:10.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 -- 172.21.15.99:0/2159309364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 msgr2=0x7f885410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 --2- 172.21.15.99:0/2159309364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 0x7f885410eb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f884c009f90 tx=0x7f884c00b330 comp rx=0 tx=0).stop 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 -- 172.21.15.99:0/2159309364 shutdown_connections 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 --2- 172.21.15.99:0/2159309364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 0x7f885410eb20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 --2- 172.21.15.99:0/2159309364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f8854103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 --2- 172.21.15.99:0/2159309364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854103b50 0x7f885410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.641+0000 7f885ac00700 1 -- 172.21.15.99:0/2159309364 >> 172.21.15.99:0/2159309364 conn(0x7f88540fca60 msgr2=0x7f88540fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.642+0000 7f885ac00700 1 -- 172.21.15.99:0/2159309364 shutdown_connections 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.642+0000 7f885ac00700 1 -- 172.21.15.99:0/2159309364 wait complete. 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.643+0000 7f885ac00700 1 Processor -- start 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.643+0000 7f885ac00700 1 -- start start 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f88541a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854103b50 0x7f88541a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 0x7f885419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:10.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8854112410 con 0x7f8854103b50 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8854112290 con 0x7f885410c660 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885ac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8854112590 con 0x7f8854103230 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f8859800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f88541a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f8859800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f88541a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:59228/0 (socket says 172.21.15.99:59228) 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f8859800700 1 -- 172.21.15.99:0/1819213857 learned_addr learned my addr 172.21.15.99:0/1819213857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f8858e00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854103b50 0x7f88541a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.644+0000 7f885a200700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 0x7f885419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 -- 172.21.15.99:0/1819213857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 msgr2=0x7f885419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f885410c660 0x7f885419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 -- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854103b50 msgr2=0x7f88541a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8854103b50 0x7f88541a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885419afc0 con 0x7f8854103230 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8859800700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f88541a06a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f884800e9f0 tx=0x7f884800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:10.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848004a30 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f884800cca0 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885419b2b0 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88480056c0 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.645+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88541a7570 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.646+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88540628e0 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.649+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f884800c7c0 con 0x7f8854103230 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8852000700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f883c0a2920 0x7f883c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:10.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f88480cede0 con 0x7f8854103230 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8858e00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f883c0a2920 0x7f883c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8852000700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] conn(0x7f883c0ac560 0x7f883c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f883c0af0d0 con 0x7f883c0ac560 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.651+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88480d1400 con 0x7f8854103230 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.652+0000 7f885a200700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] conn(0x7f883c0ac560 0x7f883c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.652+0000 7f8858e00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f883c0a2920 0x7f883c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8854102eb0 tx=0x7f8840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.652+0000 7f885a200700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] conn(0x7f883c0ac560 0x7f883c0aea00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.653+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== osd.5 v2:172.21.15.174:6808/1509311749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f883c0af0d0 con 0x7f883c0ac560 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.674+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 --> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8854062d60 con 0x7f883c0ac560 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.675+0000 7f8852000700 1 -- 172.21.15.99:0/1819213857 <== osd.5 v2:172.21.15.174:6808/1509311749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8854062d60 con 0x7f883c0ac560 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.675+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] conn(0x7f883c0ac560 msgr2=0x7f883c0aea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.675+0000 7f885ac00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] conn(0x7f883c0ac560 0x7f883c0aea00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.676+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f883c0a2920 msgr2=0x7f883c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.676+0000 7f885ac00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f883c0a2920 0x7f883c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8854102eb0 tx=0x7f8840009260 comp rx=0 tx=0).stop 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.676+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 msgr2=0x7f88541a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.676+0000 7f885ac00700 1 --2- 172.21.15.99:0/1819213857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854103230 0x7f88541a06a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f884800e9f0 tx=0x7f884800ed00 comp rx=0 tx=0).stop 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.677+0000 7f885a200700 1 -- 172.21.15.99:0/1819213857 reap_dead start 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.677+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 shutdown_connections 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.677+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 >> 172.21.15.99:0/1819213857 conn(0x7f88540fca60 msgr2=0x7f88540fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.677+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 shutdown_connections 2024-03-23T23:42:10.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:09.677+0000 7f885ac00700 1 -- 172.21.15.99:0/1819213857 wait complete. 2024-03-23T23:42:10.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:09 smithi149 ceph-mon[36083]: pgmap v348: 97 pgs: 1 activating+degraded, 1 activating, 2 peering, 93 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 2/306 objects degraded (0.654%); 0 B/s, 0 objects/s recovering 2024-03-23T23:42:10.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:09 smithi149 ceph-mon[36083]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-23T23:42:10.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:09 smithi174 ceph-mon[37002]: pgmap v348: 97 pgs: 1 activating+degraded, 1 activating, 2 peering, 93 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 2/306 objects degraded (0.654%); 0 B/s, 0 objects/s recovering 2024-03-23T23:42:10.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:09 smithi174 ceph-mon[37002]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-23T23:42:10.905 INFO:teuthology.orchestra.run.smithi099.stdout:163208757309 2024-03-23T23:42:10.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-23T23:42:11.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:10 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:42:11.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:10 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:42:11.176 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:10 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s; 126 B/s, 0 objects/s recovering 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.020 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:11 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s; 126 B/s, 0 objects/s recovering 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:11 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 op/s; 126 B/s, 0 objects/s recovering 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:11 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:12.256 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:12.270 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:12.271 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:13 smithi099 ceph-mon[31715]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:13 smithi149 ceph-mon[36083]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:14.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:13 smithi174 ceph-mon[37002]: pgmap v350: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:16.108 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:15 smithi099 ceph-mon[31715]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:15 smithi174 ceph-mon[37002]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:16.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 -- 172.21.15.99:0/96534017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 msgr2=0x7f6f58106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 --2- 172.21.15.99:0/96534017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f58106e30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f480099c0 tx=0x7f6f48009cd0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 -- 172.21.15.99:0/96534017 shutdown_connections 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 --2- 172.21.15.99:0/96534017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f5810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 --2- 172.21.15.99:0/96534017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 --2- 172.21.15.99:0/96534017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f58106e30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 -- 172.21.15.99:0/96534017 >> 172.21.15.99:0/96534017 conn(0x7f6f580792f0 msgr2=0x7f6f58079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.308+0000 7f6f60800700 1 -- 172.21.15.99:0/96534017 shutdown_connections 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.309+0000 7f6f60800700 1 -- 172.21.15.99:0/96534017 wait complete. 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.309+0000 7f6f60800700 1 Processor -- start 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 -- start start 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f58074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58112520 con 0x7f6f58107510 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581123a0 con 0x7f6f58102440 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f60800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581126a0 con 0x7f6f58102df0 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f5d400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f5de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f5d400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:59244/0 (socket says 172.21.15.99:59244) 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.310+0000 7f6f5d400700 1 -- 172.21.15.99:0/515480267 learned_addr learned my addr 172.21.15.99:0/515480267 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5e800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f58074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 -- 172.21.15.99:0/515480267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 msgr2=0x7f6f58074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 -- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 msgr2=0x7f6f58074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f58074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 -- 172.21.15.99:0/515480267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f581a6f60 con 0x7f6f58102df0 2024-03-23T23:42:16.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5de00700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5e800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f58074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f5d400700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005f50 tx=0x7f6f48003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48003680 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.311+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f480055a0 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.312+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f581a71f0 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.312+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800fda0 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.312+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f581a7640 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.313+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f58068e50 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.315+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f6f4800f460 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.317+0000 7f6f52a00700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 0x7f6f400a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.317+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f480cfb70 con 0x7f6f58102df0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.317+0000 7f6f5de00700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 0x7f6f400a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:16.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.318+0000 7f6f5de00700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 0x7f6f400a4f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009680 tx=0x7f6f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.318+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6f48013070 con 0x7f6f58102df0 2024-03-23T23:42:16.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:15 smithi149 ceph-mon[36083]: pgmap v351: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s; 126 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:16.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.600+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f5802dbf0 con 0x7f6f58102df0 2024-03-23T23:42:16.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.601+0000 7f6f52a00700 1 -- 172.21.15.99:0/515480267 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f4809cac0 con 0x7f6f58102df0 2024-03-23T23:42:16.600 INFO:teuthology.orchestra.run.smithi099.stdout:34359738457 2024-03-23T23:42:16.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 msgr2=0x7f6f400a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 0x7f6f400a4f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009680 tx=0x7f6f4c009260 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 msgr2=0x7f6f580749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005f50 tx=0x7f6f48003a90 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 shutdown_connections 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f6f400a2ac0 0x7f6f400a4f80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f58107510 0x7f6f58074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f58102440 0x7f6f58074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 --2- 172.21.15.99:0/515480267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f58102df0 0x7f6f580749d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 >> 172.21.15.99:0/515480267 conn(0x7f6f580792f0 msgr2=0x7f6f58100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:16.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 shutdown_connections 2024-03-23T23:42:16.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.604+0000 7f6f60800700 1 -- 172.21.15.99:0/515480267 wait complete. 2024-03-23T23:42:17.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:16 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/515480267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 -- 172.21.15.99:0/133165508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101020 msgr2=0x7f2bbc101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/133165508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101020 0x7f2bbc101400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009990 tx=0x7f2bac009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 -- 172.21.15.99:0/133165508 shutdown_connections 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/133165508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/133165508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101020 0x7f2bbc101400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/133165508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101940 0x7f2bbc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.679+0000 7f2bc0c00700 1 -- 172.21.15.99:0/133165508 >> 172.21.15.99:0/133165508 conn(0x7f2bbc0fca60 msgr2=0x7f2bbc0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.680+0000 7f2bc0c00700 1 -- 172.21.15.99:0/133165508 shutdown_connections 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.680+0000 7f2bc0c00700 1 -- 172.21.15.99:0/133165508 wait complete. 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 Processor -- start 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 -- start start 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101940 0x7f2bbc19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc118a10 con 0x7f2bbc106430 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc118890 con 0x7f2bbc101020 2024-03-23T23:42:17.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bc0c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc118b90 con 0x7f2bbc101940 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bbb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.681+0000 7f2bbb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:54326/0 (socket says 172.21.15.99:54326) 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 -- 172.21.15.99:0/1717082116 learned_addr learned my addr 172.21.15.99:0/1717082116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbaa00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101940 0x7f2bbc19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbbe00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 -- 172.21.15.99:0/1717082116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101940 msgr2=0x7f2bbc19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101940 0x7f2bbc19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 -- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 msgr2=0x7f2bbc1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 -- 172.21.15.99:0/1717082116 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc1a3e70 con 0x7f2bbc101020 2024-03-23T23:42:17.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbbe00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.682+0000 7f2bbb400700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bac005910 tx=0x7f2bac00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.683+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac027070 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.683+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bac003d70 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.683+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc1a4100 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.683+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f6c0 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.683+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc1ab9e0 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.684+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc068e50 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.686+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2bac02a070 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.689+0000 7f2bb3e00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 0x7f2ba00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.689+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2bac0cfa30 con 0x7f2bbc101020 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.689+0000 7f2bbaa00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 0x7f2ba00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.690+0000 7f2bbaa00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 0x7f2ba00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc106ad0 tx=0x7f2ba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:17.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.691+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2bac013070 con 0x7f2bbc101020 2024-03-23T23:42:17.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.009+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2bbc1abc10 con 0x7f2bbc101020 2024-03-23T23:42:17.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.009+0000 7f2bb3e00700 1 -- 172.21.15.99:0/1717082116 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bac09c980 con 0x7f2bbc101020 2024-03-23T23:42:17.064 INFO:teuthology.orchestra.run.smithi099.stdout:219043332147 2024-03-23T23:42:17.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.012+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 msgr2=0x7f2ba00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.012+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 0x7f2ba00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc106ad0 tx=0x7f2ba8009260 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.012+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 msgr2=0x7f2bbc19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.012+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2bac005910 tx=0x7f2bac00bab0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 shutdown_connections 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2ba00a29a0 0x7f2ba00a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bbc106430 0x7f2bbc1a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bbc101020 0x7f2bbc19ee60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 --2- 172.21.15.99:0/1717082116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bbc101940 0x7f2bbc19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 >> 172.21.15.99:0/1717082116 conn(0x7f2bbc0fca60 msgr2=0x7f2bbc103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 shutdown_connections 2024-03-23T23:42:17.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.013+0000 7f2bc0c00700 1 -- 172.21.15.99:0/1717082116 wait complete. 2024-03-23T23:42:17.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:16 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/515480267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T23:42:17.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:16 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/515480267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 -- 172.21.15.99:0/3949900722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14069e40 msgr2=0x7fcb1406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 --2- 172.21.15.99:0/3949900722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14069e40 0x7fcb1406a2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0099c0 tx=0x7fcafc009cd0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 -- 172.21.15.99:0/3949900722 shutdown_connections 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 --2- 172.21.15.99:0/3949900722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb1406a7e0 0x7fcb1410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 --2- 172.21.15.99:0/3949900722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 --2- 172.21.15.99:0/3949900722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14069e40 0x7fcb1406a2a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 -- 172.21.15.99:0/3949900722 >> 172.21.15.99:0/3949900722 conn(0x7fcb14077fc0 msgr2=0x7fcb140783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.767+0000 7fcb1a800700 1 -- 172.21.15.99:0/3949900722 shutdown_connections 2024-03-23T23:42:17.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.768+0000 7fcb1a800700 1 -- 172.21.15.99:0/3949900722 wait complete. 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.769+0000 7fcb1a800700 1 Processor -- start 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.769+0000 7fcb1a800700 1 -- start start 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 0x7fcb14102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 0x7fcb14101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb141124b0 con 0x7fcb141081e0 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14112330 con 0x7fcb14069e40 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb1a800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14112630 con 0x7fcb1406a7e0 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb13400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 0x7fcb14102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb13400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 0x7fcb14102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:54342/0 (socket says 172.21.15.99:54342) 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.770+0000 7fcb13400700 1 -- 172.21.15.99:0/229200630 learned_addr learned my addr 172.21.15.99:0/229200630 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 0x7fcb14101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb13e00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 -- 172.21.15.99:0/229200630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 msgr2=0x7fcb14102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 0x7fcb14102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 -- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 msgr2=0x7fcb141017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.771+0000 7fcb12a00700 1 -- 172.21.15.99:0/229200630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1419cb10 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.772+0000 7fcb13e00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.772+0000 7fcb12a00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 0x7fcb14101160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.773+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc027070 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.773+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcafc004050 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.773+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc022d40 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.773+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1419cda0 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.774+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1419d280 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.775+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb14103890 con 0x7fcb1406a7e0 2024-03-23T23:42:17.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.777+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcafc02a070 con 0x7fcb1406a7e0 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.779+0000 7fcb0be00700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 0x7fcaf80a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.779+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcafc0cfb40 con 0x7fcb1406a7e0 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stdout:309237645345 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.779+0000 7fcb13400700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 0x7fcaf80a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.780+0000 7fcb13400700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 0x7fcaf80a4f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005e90 tx=0x7fcb04005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:16.787+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcafc013070 con 0x7fcb1406a7e0 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.090+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fcb14068e50 con 0x7fcb1406a7e0 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.091+0000 7fcb0be00700 1 -- 172.21.15.99:0/229200630 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fcafc09ca90 con 0x7fcb1406a7e0 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 msgr2=0x7fcaf80a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 0x7fcaf80a4f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb04005e90 tx=0x7fcb04005d20 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 msgr2=0x7fcb14101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 0x7fcb14101160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098e0 tx=0x7fcafc0038b0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 shutdown_connections 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcaf80a2a70 0x7fcaf80a4f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb141081e0 0x7fcb141017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb14069e40 0x7fcb14102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 --2- 172.21.15.99:0/229200630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb1406a7e0 0x7fcb14101160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.093+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 >> 172.21.15.99:0/229200630 conn(0x7fcb14077fc0 msgr2=0x7fcb140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.094+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 shutdown_connections 2024-03-23T23:42:17.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:17.094+0000 7fcb1a800700 1 -- 172.21.15.99:0/229200630 wait complete. 2024-03-23T23:42:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:17 smithi099 ceph-mon[31715]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 102 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:17 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1717082116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T23:42:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:17 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/229200630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T23:42:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:17 smithi174 ceph-mon[37002]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 102 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:17 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1717082116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T23:42:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:17 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/229200630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T23:42:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:17 smithi149 ceph-mon[36083]: pgmap v352: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 102 B/s, 1 keys/s, 1 objects/s recovering 2024-03-23T23:42:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:17 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1717082116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T23:42:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:17 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/229200630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T23:42:19.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332147 for osd.7 2024-03-23T23:42:19.315 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:19.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645345 for osd.10 2024-03-23T23:42:19.576 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:19.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 -- 172.21.15.99:0/1155526147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea006a7e0 msgr2=0x7f1ea010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:19.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1155526147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea006a7e0 0x7f1ea010f020 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800b0d0 tx=0x7f1e9800b3e0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 -- 172.21.15.99:0/1155526147 shutdown_connections 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1155526147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea006a7e0 0x7f1ea010f020 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1155526147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea01081e0 0x7f1ea01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1155526147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0069e40 0x7f1ea006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.825+0000 7f1ea8200700 1 -- 172.21.15.99:0/1155526147 >> 172.21.15.99:0/1155526147 conn(0x7f1ea0077fc0 msgr2=0x7f1ea00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.826+0000 7f1ea8200700 1 -- 172.21.15.99:0/1155526147 shutdown_connections 2024-03-23T23:42:19.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.826+0000 7f1ea8200700 1 -- 172.21.15.99:0/1155526147 wait complete. 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.827+0000 7f1ea8200700 1 Processor -- start 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.827+0000 7f1ea8200700 1 -- start start 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 0x7f1ea019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 0x7f1ea01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:19.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0112480 con 0x7f1ea0069e40 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0112300 con 0x7f1ea006a7e0 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea8200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0112600 con 0x7f1ea01081e0 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 0x7f1ea019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 0x7f1ea019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45794/0 (socket says 172.21.15.99:45794) 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:49476/0 (socket says 172.21.15.99:49476) 2024-03-23T23:42:19.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea5800700 1 -- 172.21.15.99:0/1647218275 learned_addr learned my addr 172.21.15.99:0/1647218275 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea6200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 0x7f1ea01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 -- 172.21.15.99:0/1647218275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 msgr2=0x7f1ea01987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 0x7f1ea01987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 -- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 msgr2=0x7f1ea019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 0x7f1ea019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:19.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.828+0000 7f1ea4e00700 1 -- 172.21.15.99:0/1647218275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0198fc0 con 0x7f1ea006a7e0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1ea6200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 0x7f1ea01987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1ea4e00700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00e9f0 tx=0x7f1e8c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00c9c0 con 0x7f1ea006a7e0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e8c00cb60 con 0x7f1ea006a7e0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c018680 con 0x7f1ea006a7e0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea01992b0 con 0x7f1ea006a7e0 2024-03-23T23:42:19.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.829+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea01a5470 con 0x7f1ea006a7e0 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.830+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea0103890 con 0x7f1ea006a7e0 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.833+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f1e8c010430 con 0x7f1ea006a7e0 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.834+0000 7f1e9e000700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 0x7f1e880a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.834+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1e8c0cf6f0 con 0x7f1ea006a7e0 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.834+0000 7f1ea5800700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 0x7f1e880a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.835+0000 7f1ea5800700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 0x7f1e880a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009ca0 tx=0x7f1e94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:19.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:19.836+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1e8c026070 con 0x7f1ea006a7e0 2024-03-23T23:42:20.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:19 smithi099 ceph-mon[31715]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 93 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T23:42:20.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:19 smithi174 ceph-mon[37002]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 93 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T23:42:20.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.192+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f1ea002da30 con 0x7f1ea006a7e0 2024-03-23T23:42:20.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.193+0000 7f1e9e000700 1 -- 172.21.15.99:0/1647218275 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1e8c09c5b0 con 0x7f1ea006a7e0 2024-03-23T23:42:20.193 INFO:teuthology.orchestra.run.smithi099.stdout:343597383708 2024-03-23T23:42:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.195+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 msgr2=0x7f1e880a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.195+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 0x7f1e880a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009ca0 tx=0x7f1e94009260 comp rx=0 tx=0).stop 2024-03-23T23:42:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.195+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 msgr2=0x7f1ea019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:20.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.195+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00e9f0 tx=0x7f1e8c00ed00 comp rx=0 tx=0).stop 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 shutdown_connections 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1e880a29a0 0x7f1e880a4e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea01081e0 0x7f1ea01987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea0069e40 0x7f1ea019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 --2- 172.21.15.99:0/1647218275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea006a7e0 0x7f1ea019eb50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 >> 172.21.15.99:0/1647218275 conn(0x7f1ea0077fc0 msgr2=0x7f1ea00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:20.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 shutdown_connections 2024-03-23T23:42:20.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:20.196+0000 7f1ea8200700 1 -- 172.21.15.99:0/1647218275 wait complete. 2024-03-23T23:42:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:19 smithi149 ceph-mon[36083]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 93 B/s, 0 keys/s, 1 objects/s recovering 2024-03-23T23:42:20.766 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738457 for osd.0 2024-03-23T23:42:20.767 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:20.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:20 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1647218275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T23:42:21.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:20 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1647218275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T23:42:21.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:20 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1647218275' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T23:42:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:21 smithi174 ceph-mon[37002]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:22.220 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:21 smithi099 ceph-mon[31715]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:22.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:21 smithi149 ceph-mon[36083]: pgmap v354: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:22.736 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:24.060 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:23 smithi099 ceph-mon[31715]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:23 smithi149 ceph-mon[36083]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:24.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:23 smithi174 ceph-mon[37002]: pgmap v355: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-23T23:42:24.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383708 for osd.11 2024-03-23T23:42:24.631 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 -- 172.21.15.99:0/3192534315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c0102c80 msgr2=0x7f65c0106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 --2- 172.21.15.99:0/3192534315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c0102c80 0x7f65c0106cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009990 tx=0x7f65ac009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 -- 172.21.15.99:0/3192534315 shutdown_connections 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 --2- 172.21.15.99:0/3192534315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01073b0 0x7f65c010ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 --2- 172.21.15.99:0/3192534315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c01022d0 0x7f65c01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 --2- 172.21.15.99:0/3192534315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c0102c80 0x7f65c0106cd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 -- 172.21.15.99:0/3192534315 >> 172.21.15.99:0/3192534315 conn(0x7f65c00fdb60 msgr2=0x7f65c00fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 -- 172.21.15.99:0/3192534315 shutdown_connections 2024-03-23T23:42:25.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.328+0000 7f65c6800700 1 -- 172.21.15.99:0/3192534315 wait complete. 2024-03-23T23:42:25.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 Processor -- start 2024-03-23T23:42:25.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 -- start start 2024-03-23T23:42:25.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 0x7f65c019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 0x7f65c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01123e0 con 0x7f65c0102c80 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0112260 con 0x7f65c01073b0 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0112560 con 0x7f65c01022d0 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.330+0000 7f65c5400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34766/0 (socket says 172.21.15.99:34766) 2024-03-23T23:42:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 -- 172.21.15.99:0/3515454011 learned_addr learned my addr 172.21.15.99:0/3515454011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5e00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 0x7f65c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c4a00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 0x7f65c019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 -- 172.21.15.99:0/3515454011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 msgr2=0x7f65c01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 0x7f65c01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 -- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 msgr2=0x7f65c019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 0x7f65c019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 -- 172.21.15.99:0/3515454011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0198fa0 con 0x7f65c01022d0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5e00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 0x7f65c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c4a00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 0x7f65c019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.331+0000 7f65c5400700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f65b4012db0 tx=0x7f65b400c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.332+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b401f070 con 0x7f65c01022d0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.332+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65b401a910 con 0x7f65c01022d0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.332+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0199230 con 0x7f65c01022d0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.332+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40188e0 con 0x7f65c01022d0 2024-03-23T23:42:25.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.332+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c0074590 con 0x7f65c01022d0 2024-03-23T23:42:25.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.333+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c01037b0 con 0x7f65c01022d0 2024-03-23T23:42:25.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.335+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f65b4022070 con 0x7f65c01022d0 2024-03-23T23:42:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.336+0000 7f65be000700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 0x7f65a80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.337+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f65b40d2880 con 0x7f65c01022d0 2024-03-23T23:42:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.337+0000 7f65c4a00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 0x7f65a80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.337+0000 7f65c4a00700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 0x7f65a80a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65c0199ec0 tx=0x7f65ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:25.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.339+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f65b409f740 con 0x7f65c01022d0 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 -- 172.21.15.99:0/1519865642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074e50 msgr2=0x7f07ac112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 --2- 172.21.15.99:0/1519865642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074e50 0x7f07ac112c40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0798009990 tx=0x7f0798009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 -- 172.21.15.99:0/1519865642 shutdown_connections 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 --2- 172.21.15.99:0/1519865642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 --2- 172.21.15.99:0/1519865642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074530 0x7f07ac074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 --2- 172.21.15.99:0/1519865642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074e50 0x7f07ac112c40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.627+0000 7f07b1200700 1 -- 172.21.15.99:0/1519865642 >> 172.21.15.99:0/1519865642 conn(0x7f07ac0fdc40 msgr2=0x7f07ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.628+0000 7f07b1200700 1 -- 172.21.15.99:0/1519865642 shutdown_connections 2024-03-23T23:42:25.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.628+0000 7f07b1200700 1 -- 172.21.15.99:0/1519865642 wait complete. 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 Processor -- start 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 -- start start 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074e50 0x7f07ac19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac1a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac118af0 con 0x7f07ac113180 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac118970 con 0x7f07ac074530 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.629+0000 7f07b1200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac118c70 con 0x7f07ac074e50 2024-03-23T23:42:25.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:49528/0 (socket says 172.21.15.99:49528) 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 -- 172.21.15.99:0/1203702829 learned_addr learned my addr 172.21.15.99:0/1203702829 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9400700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074e50 0x7f07ac19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07aa800700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac1a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 -- 172.21.15.99:0/1203702829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074e50 msgr2=0x7f07ac19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074e50 0x7f07ac19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 -- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 msgr2=0x7f07ac1a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac1a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07a9e00700 1 -- 172.21.15.99:0/1203702829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac1a3f70 con 0x7f07ac074530 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.630+0000 7f07aa800700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07a9e00700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f079c00d950 tx=0x7f079c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c016070 con 0x7f07ac074530 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f079c009e20 con 0x7f07ac074530 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac1a4260 con 0x7f07ac074530 2024-03-23T23:42:25.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00f620 con 0x7f07ac074530 2024-03-23T23:42:25.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.631+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac1abae0 con 0x7f07ac074530 2024-03-23T23:42:25.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.633+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac068e50 con 0x7f07ac074530 2024-03-23T23:42:25.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.634+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f079c019070 con 0x7f07ac074530 2024-03-23T23:42:25.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.636+0000 7f07a2a00700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 0x7f07900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:25.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.636+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f079c0ce810 con 0x7f07ac074530 2024-03-23T23:42:25.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.636+0000 7f07a9400700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 0x7f07900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:25.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.637+0000 7f07a9400700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 0x7f07900a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f07ac105f30 tx=0x7f0798005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:25.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.640+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f079c09b6d0 con 0x7f07ac074530 2024-03-23T23:42:25.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.648+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f65c0068e50 con 0x7f65c01022d0 2024-03-23T23:42:25.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.649+0000 7f65be000700 1 -- 172.21.15.99:0/3515454011 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65b409f040 con 0x7f65c01022d0 2024-03-23T23:42:25.649 INFO:teuthology.orchestra.run.smithi099.stdout:253403070510 2024-03-23T23:42:25.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.652+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 msgr2=0x7f65a80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.652+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 0x7f65a80a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65c0199ec0 tx=0x7f65ac01a040 comp rx=0 tx=0).stop 2024-03-23T23:42:25.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.652+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 msgr2=0x7f65c019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.652+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f65b4012db0 tx=0x7f65b400c5b0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 shutdown_connections 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f65a80a29a0 0x7f65a80a4e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c01073b0 0x7f65c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c01022d0 0x7f65c019e5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 --2- 172.21.15.99:0/3515454011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65c0102c80 0x7f65c019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 >> 172.21.15.99:0/3515454011 conn(0x7f65c00fdb60 msgr2=0x7f65c00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 shutdown_connections 2024-03-23T23:42:25.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.653+0000 7f65c6800700 1 -- 172.21.15.99:0/3515454011 wait complete. 2024-03-23T23:42:25.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.917+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f07ac02da30 con 0x7f07ac074530 2024-03-23T23:42:25.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.918+0000 7f07a2a00700 1 -- 172.21.15.99:0/1203702829 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f079c09afd0 con 0x7f07ac074530 2024-03-23T23:42:25.918 INFO:teuthology.orchestra.run.smithi099.stdout:141733920837 2024-03-23T23:42:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.921+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 msgr2=0x7f07900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.921+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 0x7f07900a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f07ac105f30 tx=0x7f0798005fd0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.921+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 msgr2=0x7f07ac19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.921+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f079c00d950 tx=0x7f079c00dc60 comp rx=0 tx=0).stop 2024-03-23T23:42:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 shutdown_connections 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07900a2920 0x7f07900a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07ac113180 0x7f07ac1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07ac074530 0x7f07ac19ef60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 --2- 172.21.15.99:0/1203702829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07ac074e50 0x7f07ac19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 >> 172.21.15.99:0/1203702829 conn(0x7f07ac0fdc40 msgr2=0x7f07ac0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 shutdown_connections 2024-03-23T23:42:25.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:25.922+0000 7f07b1200700 1 -- 172.21.15.99:0/1203702829 wait complete. 2024-03-23T23:42:26.140 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:25 smithi099 ceph-mon[31715]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:26.140 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3515454011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T23:42:26.140 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1203702829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T23:42:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:25 smithi149 ceph-mon[36083]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3515454011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T23:42:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1203702829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T23:42:26.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:25 smithi174 ceph-mon[37002]: pgmap v356: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:26.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3515454011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T23:42:26.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1203702829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T23:42:27.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070503 got 253403070510 for osd.8 2024-03-23T23:42:27.782 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:27 smithi099 ceph-mon[31715]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:27 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1937756979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T23:42:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:27 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3892379287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 -- 172.21.15.99:0/2645016216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 msgr2=0x7ff9a41085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/2645016216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 0x7ff9a41085c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff994009990 tx=0x7ff994009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 -- 172.21.15.99:0/2645016216 shutdown_connections 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/2645016216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a406a7e0 0x7ff9a410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/2645016216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 0x7ff9a41085c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/2645016216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a4069e40 0x7ff9a406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 -- 172.21.15.99:0/2645016216 >> 172.21.15.99:0/2645016216 conn(0x7ff9a4077fc0 msgr2=0x7ff9a40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 -- 172.21.15.99:0/2645016216 shutdown_connections 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.529+0000 7ff9a9a00700 1 -- 172.21.15.99:0/2645016216 wait complete. 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.530+0000 7ff9a9a00700 1 Processor -- start 2024-03-23T23:42:28.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.530+0000 7ff9a9a00700 1 -- start start 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a406a7e0 0x7ff9a41a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 0x7ff9a419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4112490 con 0x7ff9a406a7e0 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4112310 con 0x7ff9a41081e0 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a9a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4112610 con 0x7ff9a4069e40 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34826/0 (socket says 172.21.15.99:34826) 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.531+0000 7ff9a2a00700 1 -- 172.21.15.99:0/3892379287 learned_addr learned my addr 172.21.15.99:0/3892379287 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 -- 172.21.15.99:0/3892379287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 msgr2=0x7ff9a419a990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 0x7ff9a419a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 -- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a406a7e0 msgr2=0x7ff9a41a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:28.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a406a7e0 0x7ff9a41a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 -- 172.21.15.99:0/3892379287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a419b0f0 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a2a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff994000c00 tx=0x7ff994005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994027070 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a419b380 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff994022470 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.532+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994017650 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.533+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a41a76a0 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.534+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4068e50 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.537+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7ff99402a070 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.538+0000 7ff99b400700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 0x7ff9880a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.539+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff9940cf9c0 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.539+0000 7ff9a2000700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 0x7ff9880a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.539+0000 7ff9a2000700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 0x7ff9880a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff990005950 tx=0x7ff9900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.546+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff994013070 con 0x7ff9a4069e40 2024-03-23T23:42:28.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.802+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff9a419bec0 con 0x7ff9a4069e40 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stdout:197568495673 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.803+0000 7ff99b400700 1 -- 172.21.15.99:0/3892379287 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff99409c910 con 0x7ff9a4069e40 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.805+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 msgr2=0x7ff9880a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.805+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 0x7ff9880a4e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff990005950 tx=0x7ff9900058e0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 msgr2=0x7ff9a41a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff994000c00 tx=0x7ff994005640 comp rx=0 tx=0).stop 2024-03-23T23:42:28.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 shutdown_connections 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9880a29a0 0x7ff9880a4e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9a41081e0 0x7ff9a419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a4069e40 0x7ff9a41a07d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 --2- 172.21.15.99:0/3892379287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9a406a7e0 0x7ff9a41a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 >> 172.21.15.99:0/3892379287 conn(0x7ff9a4077fc0 msgr2=0x7ff9a40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 shutdown_connections 2024-03-23T23:42:28.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.806+0000 7ff9a9a00700 1 -- 172.21.15.99:0/3892379287 wait complete. 2024-03-23T23:42:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:27 smithi149 ceph-mon[36083]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:27 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1937756979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T23:42:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:27 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3892379287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T23:42:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:27 smithi174 ceph-mon[37002]: pgmap v357: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:27 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1937756979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T23:42:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:27 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3892379287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T23:42:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:28.651 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920832 got 141733920837 for osd.4 2024-03-23T23:42:28.651 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:29 smithi099 ceph-mon[31715]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:42:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:29 smithi149 ceph-mon[36083]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:42:30.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:29 smithi174 ceph-mon[37002]: pgmap v358: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:42:31.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 -- 172.21.15.99:0/1243775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 msgr2=0x7fcc80103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1243775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc80103610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009990 tx=0x7fcc70009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 -- 172.21.15.99:0/1243775468 shutdown_connections 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1243775468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc8010c660 0x7fcc8010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1243775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc80103610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1243775468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc80103b50 0x7fcc8010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 -- 172.21.15.99:0/1243775468 >> 172.21.15.99:0/1243775468 conn(0x7fcc800fca60 msgr2=0x7fcc800fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 -- 172.21.15.99:0/1243775468 shutdown_connections 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.860+0000 7fcc7f200700 1 -- 172.21.15.99:0/1243775468 wait complete. 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 Processor -- start 2024-03-23T23:42:31.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 -- start start 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc80103b50 0x7fcc801a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 0x7fcc8019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801123f0 con 0x7fcc8010c660 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.862+0000 7fcc7f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80112270 con 0x7fcc80103b50 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80112570 con 0x7fcc80103230 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:31.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34804/0 (socket says 172.21.15.99:34804) 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 0x7fcc8019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 -- 172.21.15.99:0/1937756979 learned_addr learned my addr 172.21.15.99:0/1937756979 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7d400700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc80103b50 0x7fcc801a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 -- 172.21.15.99:0/1937756979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc80103b50 msgr2=0x7fcc801a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc80103b50 0x7fcc801a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 -- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 msgr2=0x7fcc8019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 0x7fcc8019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7de00700 1 -- 172.21.15.99:0/1937756979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8019b040 con 0x7fcc80103230 2024-03-23T23:42:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.863+0000 7fcc7e800700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 0x7fcc8019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc7de00700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc7000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70027070 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc70003d70 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000f460 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8019b2d0 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.864+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc801a75f0 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.865+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80068e50 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.869+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fcc70009d10 con 0x7fcc80103230 2024-03-23T23:42:31.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.870+0000 7fcc76a00700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 0x7fcc640a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.870+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcc700d0420 con 0x7fcc80103230 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.870+0000 7fcc7d400700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 0x7fcc640a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.871+0000 7fcc7d400700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 0x7fcc640a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc8010df80 tx=0x7fcc6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:26.871+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc70030070 con 0x7fcc80103230 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.117+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fcc801a7820 con 0x7fcc80103230 2024-03-23T23:42:31.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.117+0000 7fcc76a00700 1 -- 172.21.15.99:0/1937756979 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc70013070 con 0x7fcc80103230 2024-03-23T23:42:31.012 INFO:teuthology.orchestra.run.smithi099.stdout:287762808871 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 msgr2=0x7fcc640a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 0x7fcc640a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc8010df80 tx=0x7fcc6c009260 comp rx=0 tx=0).stop 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 msgr2=0x7fcc801a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc7000bab0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 shutdown_connections 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fcc640a29a0 0x7fcc640a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc8010c660 0x7fcc8019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc80103230 0x7fcc801a0720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 --2- 172.21.15.99:0/1937756979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc80103b50 0x7fcc801a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:31.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 >> 172.21.15.99:0/1937756979 conn(0x7fcc800fca60 msgr2=0x7fcc800fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:31.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 shutdown_connections 2024-03-23T23:42:31.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:27.121+0000 7fcc7f200700 1 -- 172.21.15.99:0/1937756979 wait complete. 2024-03-23T23:42:31.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495669 got 197568495673 for osd.6 2024-03-23T23:42:31.993 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:31 smithi099 ceph-mon[31715]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:31 smithi149 ceph-mon[36083]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:32.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:31 smithi174 ceph-mon[37002]: pgmap v359: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:33.277 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808871 for osd.9 2024-03-23T23:42:33.277 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:33 smithi099 ceph-mon[31715]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:34.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:33 smithi149 ceph-mon[36083]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:34.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:33 smithi174 ceph-mon[37002]: pgmap v360: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:42:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 -- 172.21.15.99:0/3080557507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae41081e0 msgr2=0x7f7ae41085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 --2- 172.21.15.99:0/3080557507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae41081e0 0x7f7ae41085c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40099c0 tx=0x7f7ad4009cd0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 -- 172.21.15.99:0/3080557507 shutdown_connections 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 --2- 172.21.15.99:0/3080557507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ae406a7e0 0x7f7ae410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 --2- 172.21.15.99:0/3080557507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae41081e0 0x7f7ae41085c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 --2- 172.21.15.99:0/3080557507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae4069e40 0x7f7ae406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 -- 172.21.15.99:0/3080557507 >> 172.21.15.99:0/3080557507 conn(0x7f7ae4077fc0 msgr2=0x7f7ae40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 -- 172.21.15.99:0/3080557507 shutdown_connections 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.418+0000 7f7aea400700 1 -- 172.21.15.99:0/3080557507 wait complete. 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.420+0000 7f7aea400700 1 Processor -- start 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.420+0000 7f7aea400700 1 -- start start 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:35.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 0x7f7ae41016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ae41081e0 0x7f7ae4101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4112470 con 0x7f7ae406a7e0 2024-03-23T23:42:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41122f0 con 0x7f7ae41081e0 2024-03-23T23:42:35.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7aea400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41125f0 con 0x7f7ae4069e40 2024-03-23T23:42:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:35.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39716/0 (socket says 172.21.15.99:39716) 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 -- 172.21.15.99:0/2278501863 learned_addr learned my addr 172.21.15.99:0/2278501863 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae2a00700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 0x7f7ae41016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 -- 172.21.15.99:0/2278501863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ae41081e0 msgr2=0x7f7ae4101be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ae41081e0 0x7f7ae4101be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 -- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 msgr2=0x7f7ae41016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 0x7f7ae41016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae3400700 1 -- 172.21.15.99:0/2278501863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae41a6e70 con 0x7f7ae4069e40 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.421+0000 7f7ae2a00700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 0x7f7ae41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:42:35.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7ae3400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005910 tx=0x7f7ad400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4027070 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ad4003680 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae41a7100 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41a75b0 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.422+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40173f0 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.423+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4068e50 con 0x7f7ae4069e40 2024-03-23T23:42:35.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.426+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7ad4022470 con 0x7f7ae4069e40 2024-03-23T23:42:35.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.428+0000 7f7adbe00700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 0x7f7ac80a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:35.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.428+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7ad40cf7a0 con 0x7f7ae4069e40 2024-03-23T23:42:35.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.428+0000 7f7ae2a00700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 0x7f7ac80a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:35.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.429+0000 7f7ae2a00700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 0x7f7ac80a4ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40785a0 tx=0x7f7ad0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:35.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.431+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ad4013070 con 0x7f7ae4069e40 2024-03-23T23:42:35.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.718+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7ae402dbf0 con 0x7f7ae4069e40 2024-03-23T23:42:35.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.719+0000 7f7adbe00700 1 -- 172.21.15.99:0/2278501863 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ad409c6f0 con 0x7f7ae4069e40 2024-03-23T23:42:35.722 INFO:teuthology.orchestra.run.smithi099.stdout:163208757314 2024-03-23T23:42:35.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 msgr2=0x7f7ac80a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:35.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 0x7f7ac80a4ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40785a0 tx=0x7f7ad0009260 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 msgr2=0x7f7ae4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005910 tx=0x7f7ad400f740 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 shutdown_connections 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ac80a2a20 0x7f7ac80a4ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.722+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ae41081e0 0x7f7ae4101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.723+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae4069e40 0x7f7ae4101160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.723+0000 7f7aea400700 1 --2- 172.21.15.99:0/2278501863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ae406a7e0 0x7f7ae41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:35.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.723+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 >> 172.21.15.99:0/2278501863 conn(0x7f7ae4077fc0 msgr2=0x7f7ae40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:35.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.723+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 shutdown_connections 2024-03-23T23:42:35.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:35.724+0000 7f7aea400700 1 -- 172.21.15.99:0/2278501863 wait complete. 2024-03-23T23:42:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:35 smithi099 ceph-mon[31715]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:36.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:35 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2278501863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T23:42:36.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:35 smithi149 ceph-mon[36083]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:36.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:35 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2278501863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T23:42:36.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:35 smithi174 ceph-mon[37002]: pgmap v361: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:42:36.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:35 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2278501863' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T23:42:36.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 -- 172.21.15.99:0/4248129438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd4074ce0 msgr2=0x7f7cd410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:36.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 --2- 172.21.15.99:0/4248129438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd4074ce0 0x7f7cd410e650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098f0 tx=0x7f7cc0009c00 comp rx=0 tx=0).stop 2024-03-23T23:42:36.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 -- 172.21.15.99:0/4248129438 shutdown_connections 2024-03-23T23:42:36.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 --2- 172.21.15.99:0/4248129438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd410eb90 0x7f7cd4110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 --2- 172.21.15.99:0/4248129438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd40743c0 0x7f7cd40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 --2- 172.21.15.99:0/4248129438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd4074ce0 0x7f7cd410e650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 -- 172.21.15.99:0/4248129438 >> 172.21.15.99:0/4248129438 conn(0x7f7cd40792f0 msgr2=0x7f7cd4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 -- 172.21.15.99:0/4248129438 shutdown_connections 2024-03-23T23:42:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.837+0000 7f7cdc400700 1 -- 172.21.15.99:0/4248129438 wait complete. 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.838+0000 7f7cdc400700 1 Processor -- start 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 -- start start 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd4074ce0 0x7f7cd41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd410eb90 0x7f7cd419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4114790 con 0x7f7cd4074ce0 2024-03-23T23:42:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4114610 con 0x7f7cd410eb90 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cdc400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4114910 con 0x7f7cd40743c0 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cd9a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cd9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd4074ce0 0x7f7cd41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cd9a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39730/0 (socket says 172.21.15.99:39730) 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cd9a00700 1 -- 172.21.15.99:0/1511343469 learned_addr learned my addr 172.21.15.99:0/1511343469 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.839+0000 7f7cda400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd410eb90 0x7f7cd419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 -- 172.21.15.99:0/1511343469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd410eb90 msgr2=0x7f7cd419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd410eb90 0x7f7cd419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:36.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 -- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd4074ce0 msgr2=0x7f7cd41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd4074ce0 0x7f7cd41a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 -- 172.21.15.99:0/1511343469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd419b1d0 con 0x7f7cd40743c0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd9a00700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800ec10 tx=0x7f7cc800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800cc90 con 0x7f7cd40743c0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd419b4c0 con 0x7f7cd40743c0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.840+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cc800ce30 con 0x7f7cd40743c0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.841+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8010690 con 0x7f7cd40743c0 2024-03-23T23:42:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.841+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41a7730 con 0x7f7cd40743c0 2024-03-23T23:42:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.842+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4068e50 con 0x7f7cd40743c0 2024-03-23T23:42:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.844+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7cc80438f0 con 0x7f7cd40743c0 2024-03-23T23:42:36.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.845+0000 7f7cd2000700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 0x7f7cbc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:36.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.845+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7cc80cefc0 con 0x7f7cd40743c0 2024-03-23T23:42:36.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.845+0000 7f7cd9000700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 0x7f7cbc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:36.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.846+0000 7f7cd9000700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 0x7f7cbc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009810 tx=0x7f7cc0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.847+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cc809be80 con 0x7f7cd40743c0 2024-03-23T23:42:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:36.962+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7cc8097df0 con 0x7f7cd40743c0 2024-03-23T23:42:37.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.121+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7cd419bf50 con 0x7f7cd40743c0 2024-03-23T23:42:37.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.121+0000 7f7cd2000700 1 -- 172.21.15.99:0/1511343469 <== mon.1 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cc806e5c0 con 0x7f7cd40743c0 2024-03-23T23:42:37.122 INFO:teuthology.orchestra.run.smithi099.stdout:77309411412 2024-03-23T23:42:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 msgr2=0x7f7cbc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 0x7f7cbc0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009810 tx=0x7f7cc0019040 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 msgr2=0x7f7cd41a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800ec10 tx=0x7f7cc800ef20 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 shutdown_connections 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7cbc0a2920 0x7f7cbc0a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd410eb90 0x7f7cd419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd40743c0 0x7f7cd41a0860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 --2- 172.21.15.99:0/1511343469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd4074ce0 0x7f7cd41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.124+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 >> 172.21.15.99:0/1511343469 conn(0x7f7cd40792f0 msgr2=0x7f7cd40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.125+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 shutdown_connections 2024-03-23T23:42:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.125+0000 7f7cdc400700 1 -- 172.21.15.99:0/1511343469 wait complete. 2024-03-23T23:42:37.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 -- 172.21.15.99:0/2696280484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0881081e0 msgr2=0x7fe0881085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:37.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 --2- 172.21.15.99:0/2696280484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0881081e0 0x7fe0881085c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0780098f0 tx=0x7fe078009c00 comp rx=0 tx=0).stop 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 -- 172.21.15.99:0/2696280484 shutdown_connections 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 --2- 172.21.15.99:0/2696280484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe08806a7e0 0x7fe08810f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 --2- 172.21.15.99:0/2696280484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0881081e0 0x7fe0881085c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 --2- 172.21.15.99:0/2696280484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe088069e40 0x7fe08806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 -- 172.21.15.99:0/2696280484 >> 172.21.15.99:0/2696280484 conn(0x7fe088077fc0 msgr2=0x7fe0880783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 -- 172.21.15.99:0/2696280484 shutdown_connections 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.791+0000 7fe08d600700 1 -- 172.21.15.99:0/2696280484 wait complete. 2024-03-23T23:42:37.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.792+0000 7fe08d600700 1 Processor -- start 2024-03-23T23:42:37.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.792+0000 7fe08d600700 1 -- start start 2024-03-23T23:42:37.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:37.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe08806a7e0 0x7fe0881016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 0x7fe088101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088112470 con 0x7fe0881081e0 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0881122f0 con 0x7fe088069e40 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe08d600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0881125f0 con 0x7fe08806a7e0 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe08806a7e0 0x7fe0881016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:58420/0 (socket says 172.21.15.99:58420) 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 -- 172.21.15.99:0/4017339876 learned_addr learned my addr 172.21.15.99:0/4017339876 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe087400700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 0x7fe088101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 -- 172.21.15.99:0/4017339876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe08806a7e0 msgr2=0x7fe0881016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe08806a7e0 0x7fe0881016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 -- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 msgr2=0x7fe088101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:37.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 0x7fe088101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:37.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe086a00700 1 -- 172.21.15.99:0/4017339876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0881a6e90 con 0x7fe088069e40 2024-03-23T23:42:37.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.793+0000 7fe087400700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 0x7fe088101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:37.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe086a00700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe078009fd0 tx=0x7fe0780094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:37.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078025070 con 0x7fe088069e40 2024-03-23T23:42:37.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe07800ba40 con 0x7fe088069e40 2024-03-23T23:42:37.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0881a7120 con 0x7fe088069e40 2024-03-23T23:42:37.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078004c00 con 0x7fe088069e40 2024-03-23T23:42:37.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.794+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0881a75d0 con 0x7fe088069e40 2024-03-23T23:42:37.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.795+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088068e50 con 0x7fe088069e40 2024-03-23T23:42:37.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.798+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe078028070 con 0x7fe088069e40 2024-03-23T23:42:37.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.800+0000 7fe07f400700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 0x7fe06c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:37.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.800+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0780cf2c0 con 0x7fe088069e40 2024-03-23T23:42:37.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.800+0000 7fe086000700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 0x7fe06c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:37.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.801+0000 7fe086000700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 0x7fe06c0a4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe088078630 tx=0x7fe074009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:37.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:37.801+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe078013030 con 0x7fe088069e40 2024-03-23T23:42:37.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757309 got 163208757314 for osd.5 2024-03-23T23:42:37.915 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:38.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.074+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe08802da30 con 0x7fe088069e40 2024-03-23T23:42:38.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.074+0000 7fe07f400700 1 -- 172.21.15.99:0/4017339876 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe07809c370 con 0x7fe088069e40 2024-03-23T23:42:38.075 INFO:teuthology.orchestra.run.smithi099.stdout:107374182476 2024-03-23T23:42:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 msgr2=0x7fe06c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 0x7fe06c0a4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe088078630 tx=0x7fe074009310 comp rx=0 tx=0).stop 2024-03-23T23:42:38.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 msgr2=0x7fe088101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe078009fd0 tx=0x7fe0780094b0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 shutdown_connections 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe06c0a2910 0x7fe06c0a4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0881081e0 0x7fe088101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe088069e40 0x7fe088101160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 --2- 172.21.15.99:0/4017339876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe08806a7e0 0x7fe0881016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.077+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 >> 172.21.15.99:0/4017339876 conn(0x7fe088077fc0 msgr2=0x7fe0880ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.078+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 shutdown_connections 2024-03-23T23:42:38.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.078+0000 7fe08d600700 1 -- 172.21.15.99:0/4017339876 wait complete. 2024-03-23T23:42:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:37 smithi099 ceph-mon[31715]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:37 smithi099 ceph-mon[31715]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-23T23:42:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:37 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1511343469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T23:42:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 -- 172.21.15.99:0/3819572016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102440 msgr2=0x7f9f10102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 --2- 172.21.15.99:0/3819572016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102440 0x7f9f10102820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f000099c0 tx=0x7f9f00009cd0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 -- 172.21.15.99:0/3819572016 shutdown_connections 2024-03-23T23:42:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 --2- 172.21.15.99:0/3819572016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10107510 0x7f9f1010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 --2- 172.21.15.99:0/3819572016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102440 0x7f9f10102820 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 --2- 172.21.15.99:0/3819572016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10102df0 0x7f9f10106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 -- 172.21.15.99:0/3819572016 >> 172.21.15.99:0/3819572016 conn(0x7f9f100792f0 msgr2=0x7f9f10079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.186+0000 7f9f15200700 1 -- 172.21.15.99:0/3819572016 shutdown_connections 2024-03-23T23:42:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.187+0000 7f9f15200700 1 -- 172.21.15.99:0/3819572016 wait complete. 2024-03-23T23:42:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 Processor -- start 2024-03-23T23:42:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 -- start start 2024-03-23T23:42:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 0x7f9f1019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10107510 0x7f9f10198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10112570 con 0x7f9f10107510 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101123f0 con 0x7f9f10102440 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f15200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f101126f0 con 0x7f9f10102df0 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f0de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 0x7f9f1019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f07e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f0de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 0x7f9f1019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:58450/0 (socket says 172.21.15.99:58450) 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f07e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39758/0 (socket says 172.21.15.99:39758) 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f0de00700 1 -- 172.21.15.99:0/3141194732 learned_addr learned my addr 172.21.15.99:0/3141194732 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.188+0000 7f9f0e800700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10107510 0x7f9f10198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 -- 172.21.15.99:0/3141194732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 msgr2=0x7f9f1019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 0x7f9f1019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 -- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10107510 msgr2=0x7f9f10198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10107510 0x7f9f10198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 -- 172.21.15.99:0/3141194732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10199010 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f07e00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9c0 tx=0x7f9ef800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.189+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800c9c0 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.190+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ef800cb60 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.190+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8018680 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.190+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f10199300 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.190+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f101a5510 con 0x7f9f10102df0 2024-03-23T23:42:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.191+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f10103910 con 0x7f9f10102df0 2024-03-23T23:42:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.194+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9ef8010430 con 0x7f9f10102df0 2024-03-23T23:42:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.196+0000 7f9f06a00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 0x7f9ef40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.196+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9ef80ceb90 con 0x7f9f10102df0 2024-03-23T23:42:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.196+0000 7f9f0de00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 0x7f9ef40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.197+0000 7f9f0de00700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 0x7f9ef40a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.204+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ef809bbb0 con 0x7f9f10102df0 2024-03-23T23:42:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:37 smithi149 ceph-mon[36083]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:37 smithi149 ceph-mon[36083]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-23T23:42:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:37 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1511343469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T23:42:38.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:37 smithi174 ceph-mon[37002]: pgmap v362: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:38.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:37 smithi174 ceph-mon[37002]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-23T23:42:38.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:37 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1511343469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T23:42:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.465+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f1002da30 con 0x7f9f10102df0 2024-03-23T23:42:38.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.465+0000 7f9f06a00700 1 -- 172.21.15.99:0/3141194732 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ef809b4b0 con 0x7f9f10102df0 2024-03-23T23:42:38.465 INFO:teuthology.orchestra.run.smithi099.stdout:55834574937 2024-03-23T23:42:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 msgr2=0x7f9ef40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 0x7f9ef40a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00005fb0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 msgr2=0x7f9f1019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9c0 tx=0x7f9ef800ecd0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 shutdown_connections 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9ef40a2910 0x7f9ef40a4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f10107510 0x7f9f10198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f10102440 0x7f9f1019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 --2- 172.21.15.99:0/3141194732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f10102df0 0x7f9f1019ebf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 >> 172.21.15.99:0/3141194732 conn(0x7f9f100792f0 msgr2=0x7f9f101060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:38.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.469+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 shutdown_connections 2024-03-23T23:42:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:38.470+0000 7f9f15200700 1 -- 172.21.15.99:0/3141194732 wait complete. 2024-03-23T23:42:39.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411412 for osd.2 2024-03-23T23:42:39.052 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:39.336 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:38 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4017339876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T23:42:39.336 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:38 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3141194732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T23:42:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:38 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4017339876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T23:42:39.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:38 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3141194732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T23:42:39.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:38 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4017339876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T23:42:39.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:38 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3141194732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T23:42:40.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574937 for osd.1 2024-03-23T23:42:40.057 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:39 smithi099 ceph-mon[31715]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:42:40.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:39 smithi149 ceph-mon[36083]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:42:40.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:39 smithi174 ceph-mon[37002]: pgmap v363: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:42:40.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182476 for osd.3 2024-03-23T23:42:40.466 DEBUG:teuthology.parallel:result is None 2024-03-23T23:42:40.466 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-23T23:42:40.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T23:42:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:41 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:41.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:41 smithi099 ceph-mon[31715]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:41.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:41.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:41 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:41.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:41 smithi174 ceph-mon[37002]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:41 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:42.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:41 smithi149 ceph-mon[36083]: pgmap v364: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:42.840 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:42:42.841 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:42:42.841 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:42:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:42:42.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 -- 172.21.15.99:0/3545746465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f013006a7e0 msgr2=0x7f013010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 --2- 172.21.15.99:0/3545746465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f013006a7e0 0x7f013010f020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0128009f90 tx=0x7f012800b330 comp rx=0 tx=0).stop 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 -- 172.21.15.99:0/3545746465 shutdown_connections 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 --2- 172.21.15.99:0/3545746465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f013006a7e0 0x7f013010f020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 --2- 172.21.15.99:0/3545746465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01301081e0 0x7f01301085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 --2- 172.21.15.99:0/3545746465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0130069e40 0x7f013006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 -- 172.21.15.99:0/3545746465 >> 172.21.15.99:0/3545746465 conn(0x7f0130077fc0 msgr2=0x7f01300783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:43.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 -- 172.21.15.99:0/3545746465 shutdown_connections 2024-03-23T23:42:43.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.021+0000 7f0137a00700 1 -- 172.21.15.99:0/3545746465 wait complete. 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.022+0000 7f0137a00700 1 Processor -- start 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.022+0000 7f0137a00700 1 -- start start 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 0x7f01301966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 0x7f0130196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01301081e0 0x7f013019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130110360 con 0x7f0130069e40 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301101e0 con 0x7f01301081e0 2024-03-23T23:42:43.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0137a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301104e0 con 0x7f013006a7e0 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0135000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 0x7f01301966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0135000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 0x7f01301966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40882/0 (socket says 172.21.15.99:40882) 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0135000700 1 -- 172.21.15.99:0/3809947964 learned_addr learned my addr 172.21.15.99:0/3809947964 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 0x7f0130196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f0135a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01301081e0 0x7f013019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 -- 172.21.15.99:0/3809947964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01301081e0 msgr2=0x7f013019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01301081e0 0x7f013019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 -- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 msgr2=0x7f01301966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 0x7f01301966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.023+0000 7f012fe00700 1 -- 172.21.15.99:0/3809947964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013019b700 con 0x7f013006a7e0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f012fe00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 0x7f0130196c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e960 tx=0x7f011c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c004020 con 0x7f013006a7e0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f011c009950 con 0x7f013006a7e0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013019b9f0 con 0x7f013006a7e0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c003d20 con 0x7f013006a7e0 2024-03-23T23:42:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.024+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01301a3270 con 0x7f013006a7e0 2024-03-23T23:42:43.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.025+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0130068e50 con 0x7f013006a7e0 2024-03-23T23:42:43.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.029+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f011c01f020 con 0x7f013006a7e0 2024-03-23T23:42:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.030+0000 7f012d600700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 0x7f01180a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.030+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f011c0d1d30 con 0x7f013006a7e0 2024-03-23T23:42:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.030+0000 7f0135000700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 0x7f01180a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:43.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.031+0000 7f0135000700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 0x7f01180a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0124009ca0 tx=0x7f0124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:43.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.034+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f011c026060 con 0x7f013006a7e0 2024-03-23T23:42:43.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:42:43.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:42:43.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:42:43.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.294+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01300625d0 con 0x7f01180a2910 2024-03-23T23:42:43.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.318+0000 7f012d600700 1 -- 172.21.15.99:0/3809947964 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259305 (secure 0 0 0) 0x7f01300625d0 con 0x7f01180a2910 2024-03-23T23:42:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.322+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 msgr2=0x7f01180a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.322+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 0x7f01180a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0124009ca0 tx=0x7f0124009260 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.322+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 msgr2=0x7f0130196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.322+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 0x7f0130196c30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e960 tx=0x7f011c00ec70 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 shutdown_connections 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01180a2910 0x7f01180a4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01301081e0 0x7f013019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0130069e40 0x7f01301966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 --2- 172.21.15.99:0/3809947964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f013006a7e0 0x7f0130196c30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 >> 172.21.15.99:0/3809947964 conn(0x7f0130077fc0 msgr2=0x7f01300ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 shutdown_connections 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:43.323+0000 7f0137a00700 1 -- 172.21.15.99:0/3809947964 wait complete. 2024-03-23T23:42:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:42:43.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:43 smithi174 ceph-mon[37002]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T23:42:44.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:43 smithi099 ceph-mon[31715]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T23:42:44.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:43 smithi149 ceph-mon[36083]: pgmap v365: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-23T23:42:44.313 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-03-23T23:42:42.758262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660888,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":990,"num_read_kb":622,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381664,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560160,"statfs":{"total":1151940427776,"available":1151549603840,"internally_reserved":0,"allocated":11993088,"data_stored":7452324,"data_compressed":181782,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":58,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004213"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751476+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:42:36.751476+0000","last_peered":"2024-03-23T23:42:36.751476+0000","last_clean":"2024-03-23T23:42:36.751476+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:42:36.751476+0000","last_undegraded":"2024-03-23T23:42:36.751476+0000","last_fullsized":"2024-03-23T23:42:36.751476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758307+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:42:36.758307+0000","last_peered":"2024-03-23T23:42:36.758307+0000","last_clean":"2024-03-23T23:42:36.758307+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:42:36.758307+0000","last_undegraded":"2024-03-23T23:42:36.758307+0000","last_fullsized":"2024-03-23T23:42:36.758307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761253+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:42:36.761253+0000","last_peered":"2024-03-23T23:42:36.761253+0000","last_clean":"2024-03-23T23:42:36.761253+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:42:36.761253+0000","last_undegraded":"2024-03-23T23:42:36.761253+0000","last_fullsized":"2024-03-23T23:42:36.761253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722922+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:42:36.722922+0000","last_peered":"2024-03-23T23:42:36.722922+0000","last_clean":"2024-03-23T23:42:36.722922+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:42:36.722922+0000","last_undegraded":"2024-03-23T23:42:36.722922+0000","last_fullsized":"2024-03-23T23:42:36.722922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724866+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:42:36.724866+0000","last_peered":"2024-03-23T23:42:36.724866+0000","last_clean":"2024-03-23T23:42:36.724866+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:42:36.724866+0000","last_undegraded":"2024-03-23T23:42:36.724866+0000","last_fullsized":"2024-03-23T23:42:36.724866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724820+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:42:36.724820+0000","last_peered":"2024-03-23T23:42:36.724820+0000","last_clean":"2024-03-23T23:42:36.724820+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:42:36.724820+0000","last_undegraded":"2024-03-23T23:42:36.724820+0000","last_fullsized":"2024-03-23T23:42:36.724820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.747830+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:42:36.747830+0000","last_peered":"2024-03-23T23:42:36.747830+0000","last_clean":"2024-03-23T23:42:36.747830+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:42:36.747830+0000","last_undegraded":"2024-03-23T23:42:36.747830+0000","last_fullsized":"2024-03-23T23:42:36.747830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738623+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:42:36.738623+0000","last_peered":"2024-03-23T23:42:36.738623+0000","last_clean":"2024-03-23T23:42:36.738623+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:42:36.738623+0000","last_undegraded":"2024-03-23T23:42:36.738623+0000","last_fullsized":"2024-03-23T23:42:36.738623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728376+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:42:36.728376+0000","last_peered":"2024-03-23T23:42:36.728376+0000","last_clean":"2024-03-23T23:42:36.728376+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:42:36.728376+0000","last_undegraded":"2024-03-23T23:42:36.728376+0000","last_fullsized":"2024-03-23T23:42:36.728376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.723262+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:42:36.723262+0000","last_peered":"2024-03-23T23:42:36.723262+0000","last_clean":"2024-03-23T23:42:36.723262+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:42:36.723262+0000","last_undegraded":"2024-03-23T23:42:36.723262+0000","last_fullsized":"2024-03-23T23:42:36.723262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.720783+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:42:36.720783+0000","last_peered":"2024-03-23T23:42:36.720783+0000","last_clean":"2024-03-23T23:42:36.720783+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:42:36.720783+0000","last_undegraded":"2024-03-23T23:42:36.720783+0000","last_fullsized":"2024-03-23T23:42:36.720783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.715456+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:42:36.715456+0000","last_peered":"2024-03-23T23:42:36.715456+0000","last_clean":"2024-03-23T23:42:36.715456+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:42:36.715456+0000","last_undegraded":"2024-03-23T23:42:36.715456+0000","last_fullsized":"2024-03-23T23:42:36.715456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.746606+0000","last_change":"2024-03-23T23:42:07.892099+0000","last_active":"2024-03-23T23:42:36.746606+0000","last_peered":"2024-03-23T23:42:36.746606+0000","last_clean":"2024-03-23T23:42:36.746606+0000","last_became_active":"2024-03-23T23:42:07.891702+0000","last_became_peered":"2024-03-23T23:42:07.891702+0000","last_unstale":"2024-03-23T23:42:36.746606+0000","last_undegraded":"2024-03-23T23:42:36.746606+0000","last_fullsized":"2024-03-23T23:42:36.746606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:52:44.779482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.759908+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:42:36.759908+0000","last_peered":"2024-03-23T23:42:36.759908+0000","last_clean":"2024-03-23T23:42:36.759908+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:42:36.759908+0000","last_undegraded":"2024-03-23T23:42:36.759908+0000","last_fullsized":"2024-03-23T23:42:36.759908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758585+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:42:36.758585+0000","last_peered":"2024-03-23T23:42:36.758585+0000","last_clean":"2024-03-23T23:42:36.758585+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:42:36.758585+0000","last_undegraded":"2024-03-23T23:42:36.758585+0000","last_fullsized":"2024-03-23T23:42:36.758585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758550+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:42:36.758550+0000","last_peered":"2024-03-23T23:42:36.758550+0000","last_clean":"2024-03-23T23:42:36.758550+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:42:36.758550+0000","last_undegraded":"2024-03-23T23:42:36.758550+0000","last_fullsized":"2024-03-23T23:42:36.758550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751597+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:42:36.751597+0000","last_peered":"2024-03-23T23:42:36.751597+0000","last_clean":"2024-03-23T23:42:36.751597+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:42:36.751597+0000","last_undegraded":"2024-03-23T23:42:36.751597+0000","last_fullsized":"2024-03-23T23:42:36.751597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749275+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:42:36.749275+0000","last_peered":"2024-03-23T23:42:36.749275+0000","last_clean":"2024-03-23T23:42:36.749275+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:42:36.749275+0000","last_undegraded":"2024-03-23T23:42:36.749275+0000","last_fullsized":"2024-03-23T23:42:36.749275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.721704+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:42:36.721704+0000","last_peered":"2024-03-23T23:42:36.721704+0000","last_clean":"2024-03-23T23:42:36.721704+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:42:36.721704+0000","last_undegraded":"2024-03-23T23:42:36.721704+0000","last_fullsized":"2024-03-23T23:42:36.721704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:37.398545+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:42:37.398545+0000","last_peered":"2024-03-23T23:42:37.398545+0000","last_clean":"2024-03-23T23:42:37.398545+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:42:37.398545+0000","last_undegraded":"2024-03-23T23:42:37.398545+0000","last_fullsized":"2024-03-23T23:42:37.398545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740593+0000","last_change":"2024-03-23T23:42:07.890419+0000","last_active":"2024-03-23T23:42:36.740593+0000","last_peered":"2024-03-23T23:42:36.740593+0000","last_clean":"2024-03-23T23:42:36.740593+0000","last_became_active":"2024-03-23T23:42:07.889998+0000","last_became_peered":"2024-03-23T23:42:07.889998+0000","last_unstale":"2024-03-23T23:42:36.740593+0000","last_undegraded":"2024-03-23T23:42:36.740593+0000","last_fullsized":"2024-03-23T23:42:36.740593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:35:13.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.672737+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:42:08.672737+0000","last_peered":"2024-03-23T23:42:08.672737+0000","last_clean":"2024-03-23T23:42:08.672737+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:42:08.672737+0000","last_undegraded":"2024-03-23T23:42:08.672737+0000","last_fullsized":"2024-03-23T23:42:08.672737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761400+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:42:36.761400+0000","last_peered":"2024-03-23T23:42:36.761400+0000","last_clean":"2024-03-23T23:42:36.761400+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:42:36.761400+0000","last_undegraded":"2024-03-23T23:42:36.761400+0000","last_fullsized":"2024-03-23T23:42:36.761400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.892818+0000","last_change":"2024-03-23T23:42:07.892818+0000","last_active":"2024-03-23T23:42:07.892818+0000","last_peered":"2024-03-23T23:42:07.892818+0000","last_clean":"2024-03-23T23:42:07.892818+0000","last_became_active":"2024-03-23T23:42:07.880207+0000","last_became_peered":"2024-03-23T23:42:07.880207+0000","last_unstale":"2024-03-23T23:42:07.892818+0000","last_undegraded":"2024-03-23T23:42:07.892818+0000","last_fullsized":"2024-03-23T23:42:07.892818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:04:54.088457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743533+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:42:36.743533+0000","last_peered":"2024-03-23T23:42:36.743533+0000","last_clean":"2024-03-23T23:42:36.743533+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:42:36.743533+0000","last_undegraded":"2024-03-23T23:42:36.743533+0000","last_fullsized":"2024-03-23T23:42:36.743533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876259+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:42:07.876259+0000","last_peered":"2024-03-23T23:42:07.876259+0000","last_clean":"2024-03-23T23:42:07.876259+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:42:07.876259+0000","last_undegraded":"2024-03-23T23:42:07.876259+0000","last_fullsized":"2024-03-23T23:42:07.876259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727324+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:42:36.727324+0000","last_peered":"2024-03-23T23:42:36.727324+0000","last_clean":"2024-03-23T23:42:36.727324+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:42:36.727324+0000","last_undegraded":"2024-03-23T23:42:36.727324+0000","last_fullsized":"2024-03-23T23:42:36.727324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":121,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:37.411098+0000","last_change":"2024-03-23T23:42:08.388477+0000","last_active":"2024-03-23T23:42:37.411098+0000","last_peered":"2024-03-23T23:42:37.411098+0000","last_clean":"2024-03-23T23:42:37.411098+0000","last_became_active":"2024-03-23T23:42:08.384694+0000","last_became_peered":"2024-03-23T23:42:08.384694+0000","last_unstale":"2024-03-23T23:42:37.411098+0000","last_undegraded":"2024-03-23T23:42:37.411098+0000","last_fullsized":"2024-03-23T23:42:37.411098+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:13.778147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.718444+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.718444+0000","last_peered":"2024-03-23T23:42:36.718444+0000","last_clean":"2024-03-23T23:42:36.718444+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:42:36.718444+0000","last_undegraded":"2024-03-23T23:42:36.718444+0000","last_fullsized":"2024-03-23T23:42:36.718444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.882733+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:42:07.882733+0000","last_peered":"2024-03-23T23:42:07.882733+0000","last_clean":"2024-03-23T23:42:07.882733+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:42:07.882733+0000","last_undegraded":"2024-03-23T23:42:07.882733+0000","last_fullsized":"2024-03-23T23:42:07.882733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743447+0000","last_change":"2024-03-23T23:42:08.673791+0000","last_active":"2024-03-23T23:42:36.743447+0000","last_peered":"2024-03-23T23:42:36.743447+0000","last_clean":"2024-03-23T23:42:36.743447+0000","last_became_active":"2024-03-23T23:42:08.673494+0000","last_became_peered":"2024-03-23T23:42:08.673494+0000","last_unstale":"2024-03-23T23:42:36.743447+0000","last_undegraded":"2024-03-23T23:42:36.743447+0000","last_fullsized":"2024-03-23T23:42:36.743447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:22:10.209266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.877485+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:42:07.877485+0000","last_peered":"2024-03-23T23:42:07.877485+0000","last_clean":"2024-03-23T23:42:07.877485+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:42:07.877485+0000","last_undegraded":"2024-03-23T23:42:07.877485+0000","last_fullsized":"2024-03-23T23:42:07.877485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738460+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:42:36.738460+0000","last_peered":"2024-03-23T23:42:36.738460+0000","last_clean":"2024-03-23T23:42:36.738460+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:42:36.738460+0000","last_undegraded":"2024-03-23T23:42:36.738460+0000","last_fullsized":"2024-03-23T23:42:36.738460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.880586+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:42:07.880586+0000","last_peered":"2024-03-23T23:42:07.880586+0000","last_clean":"2024-03-23T23:42:07.880586+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:42:07.880586+0000","last_undegraded":"2024-03-23T23:42:07.880586+0000","last_fullsized":"2024-03-23T23:42:07.880586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749958+0000","last_change":"2024-03-23T23:42:07.891565+0000","last_active":"2024-03-23T23:42:36.749958+0000","last_peered":"2024-03-23T23:42:36.749958+0000","last_clean":"2024-03-23T23:42:36.749958+0000","last_became_active":"2024-03-23T23:42:07.890328+0000","last_became_peered":"2024-03-23T23:42:07.890328+0000","last_unstale":"2024-03-23T23:42:36.749958+0000","last_undegraded":"2024-03-23T23:42:36.749958+0000","last_fullsized":"2024-03-23T23:42:36.749958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:44:07.467469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:40.377397+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:42:40.377397+0000","last_peered":"2024-03-23T23:42:40.377397+0000","last_clean":"2024-03-23T23:42:40.377397+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:42:40.377397+0000","last_undegraded":"2024-03-23T23:42:40.377397+0000","last_fullsized":"2024-03-23T23:42:40.377397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742341+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:42:36.742341+0000","last_peered":"2024-03-23T23:42:36.742341+0000","last_clean":"2024-03-23T23:42:36.742341+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:42:36.742341+0000","last_undegraded":"2024-03-23T23:42:36.742341+0000","last_fullsized":"2024-03-23T23:42:36.742341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878070+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:42:07.878070+0000","last_peered":"2024-03-23T23:42:07.878070+0000","last_clean":"2024-03-23T23:42:07.878070+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:42:07.878070+0000","last_undegraded":"2024-03-23T23:42:07.878070+0000","last_fullsized":"2024-03-23T23:42:07.878070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760716+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:42:36.760716+0000","last_peered":"2024-03-23T23:42:36.760716+0000","last_clean":"2024-03-23T23:42:36.760716+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:42:36.760716+0000","last_undegraded":"2024-03-23T23:42:36.760716+0000","last_fullsized":"2024-03-23T23:42:36.760716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881789+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:42:07.881789+0000","last_peered":"2024-03-23T23:42:07.881789+0000","last_clean":"2024-03-23T23:42:07.881789+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:42:07.881789+0000","last_undegraded":"2024-03-23T23:42:07.881789+0000","last_fullsized":"2024-03-23T23:42:07.881789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728388+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:42:36.728388+0000","last_peered":"2024-03-23T23:42:36.728388+0000","last_clean":"2024-03-23T23:42:36.728388+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:42:36.728388+0000","last_undegraded":"2024-03-23T23:42:36.728388+0000","last_fullsized":"2024-03-23T23:42:36.728388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":230,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:40.413517+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:42:40.413517+0000","last_peered":"2024-03-23T23:42:40.413517+0000","last_clean":"2024-03-23T23:42:40.413517+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:42:40.413517+0000","last_undegraded":"2024-03-23T23:42:40.413517+0000","last_fullsized":"2024-03-23T23:42:40.413517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62800,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":149,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725831+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:42:36.725831+0000","last_peered":"2024-03-23T23:42:36.725831+0000","last_clean":"2024-03-23T23:42:36.725831+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:42:36.725831+0000","last_undegraded":"2024-03-23T23:42:36.725831+0000","last_fullsized":"2024-03-23T23:42:36.725831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878290+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:42:07.878290+0000","last_peered":"2024-03-23T23:42:07.878290+0000","last_clean":"2024-03-23T23:42:07.878290+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:42:07.878290+0000","last_undegraded":"2024-03-23T23:42:07.878290+0000","last_fullsized":"2024-03-23T23:42:07.878290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740229+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:42:36.740229+0000","last_peered":"2024-03-23T23:42:36.740229+0000","last_clean":"2024-03-23T23:42:36.740229+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:42:36.740229+0000","last_undegraded":"2024-03-23T23:42:36.740229+0000","last_fullsized":"2024-03-23T23:42:36.740229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881072+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:42:07.881072+0000","last_peered":"2024-03-23T23:42:07.881072+0000","last_clean":"2024-03-23T23:42:07.881072+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:42:07.881072+0000","last_undegraded":"2024-03-23T23:42:07.881072+0000","last_fullsized":"2024-03-23T23:42:07.881072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.736988+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:42:36.736988+0000","last_peered":"2024-03-23T23:42:36.736988+0000","last_clean":"2024-03-23T23:42:36.736988+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:42:36.736988+0000","last_undegraded":"2024-03-23T23:42:36.736988+0000","last_fullsized":"2024-03-23T23:42:36.736988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:37.400717+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:42:37.400717+0000","last_peered":"2024-03-23T23:42:37.400717+0000","last_clean":"2024-03-23T23:42:37.400717+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:42:37.400717+0000","last_undegraded":"2024-03-23T23:42:37.400717+0000","last_fullsized":"2024-03-23T23:42:37.400717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725306+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:42:36.725306+0000","last_peered":"2024-03-23T23:42:36.725306+0000","last_clean":"2024-03-23T23:42:36.725306+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:42:36.725306+0000","last_undegraded":"2024-03-23T23:42:36.725306+0000","last_fullsized":"2024-03-23T23:42:36.725306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.875951+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:42:07.875951+0000","last_peered":"2024-03-23T23:42:07.875951+0000","last_clean":"2024-03-23T23:42:07.875951+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:42:07.875951+0000","last_undegraded":"2024-03-23T23:42:07.875951+0000","last_fullsized":"2024-03-23T23:42:07.875951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726634+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:42:36.726634+0000","last_peered":"2024-03-23T23:42:36.726634+0000","last_clean":"2024-03-23T23:42:36.726634+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:42:36.726634+0000","last_undegraded":"2024-03-23T23:42:36.726634+0000","last_fullsized":"2024-03-23T23:42:36.726634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876451+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:42:07.876451+0000","last_peered":"2024-03-23T23:42:07.876451+0000","last_clean":"2024-03-23T23:42:07.876451+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:42:07.876451+0000","last_undegraded":"2024-03-23T23:42:07.876451+0000","last_fullsized":"2024-03-23T23:42:07.876451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722437+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:42:36.722437+0000","last_peered":"2024-03-23T23:42:36.722437+0000","last_clean":"2024-03-23T23:42:36.722437+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:42:36.722437+0000","last_undegraded":"2024-03-23T23:42:36.722437+0000","last_fullsized":"2024-03-23T23:42:36.722437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.888190+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:42:07.888190+0000","last_peered":"2024-03-23T23:42:07.888190+0000","last_clean":"2024-03-23T23:42:07.888190+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:42:07.888190+0000","last_undegraded":"2024-03-23T23:42:07.888190+0000","last_fullsized":"2024-03-23T23:42:07.888190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728326+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:42:36.728326+0000","last_peered":"2024-03-23T23:42:36.728326+0000","last_clean":"2024-03-23T23:42:36.728326+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:42:36.728326+0000","last_undegraded":"2024-03-23T23:42:36.728326+0000","last_fullsized":"2024-03-23T23:42:36.728326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:38.400317+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:42:38.400317+0000","last_peered":"2024-03-23T23:42:38.400317+0000","last_clean":"2024-03-23T23:42:38.400317+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:42:38.400317+0000","last_undegraded":"2024-03-23T23:42:38.400317+0000","last_fullsized":"2024-03-23T23:42:38.400317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.757668+0000","last_change":"2024-03-23T23:42:07.890902+0000","last_active":"2024-03-23T23:42:36.757668+0000","last_peered":"2024-03-23T23:42:36.757668+0000","last_clean":"2024-03-23T23:42:36.757668+0000","last_became_active":"2024-03-23T23:42:07.890514+0000","last_became_peered":"2024-03-23T23:42:07.890514+0000","last_unstale":"2024-03-23T23:42:36.757668+0000","last_undegraded":"2024-03-23T23:42:36.757668+0000","last_fullsized":"2024-03-23T23:42:36.757668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:42.343732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725916+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:42:36.725916+0000","last_peered":"2024-03-23T23:42:36.725916+0000","last_clean":"2024-03-23T23:42:36.725916+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:42:36.725916+0000","last_undegraded":"2024-03-23T23:42:36.725916+0000","last_fullsized":"2024-03-23T23:42:36.725916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:40.404140+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:42:40.404140+0000","last_peered":"2024-03-23T23:42:40.404140+0000","last_clean":"2024-03-23T23:42:40.404140+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:42:40.404140+0000","last_undegraded":"2024-03-23T23:42:40.404140+0000","last_fullsized":"2024-03-23T23:42:40.404140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726081+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:42:36.726081+0000","last_peered":"2024-03-23T23:42:36.726081+0000","last_clean":"2024-03-23T23:42:36.726081+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:42:36.726081+0000","last_undegraded":"2024-03-23T23:42:36.726081+0000","last_fullsized":"2024-03-23T23:42:36.726081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360541+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:42:08.360541+0000","last_peered":"2024-03-23T23:42:08.360541+0000","last_clean":"2024-03-23T23:42:08.360541+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:42:08.360541+0000","last_undegraded":"2024-03-23T23:42:08.360541+0000","last_fullsized":"2024-03-23T23:42:08.360541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878026+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:42:07.878026+0000","last_peered":"2024-03-23T23:42:07.878026+0000","last_clean":"2024-03-23T23:42:07.878026+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:42:07.878026+0000","last_undegraded":"2024-03-23T23:42:07.878026+0000","last_fullsized":"2024-03-23T23:42:07.878026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729760+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:42:36.729760+0000","last_peered":"2024-03-23T23:42:36.729760+0000","last_clean":"2024-03-23T23:42:36.729760+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:42:36.729760+0000","last_undegraded":"2024-03-23T23:42:36.729760+0000","last_fullsized":"2024-03-23T23:42:36.729760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.895117+0000","last_change":"2024-03-23T23:42:07.895117+0000","last_active":"2024-03-23T23:42:07.895117+0000","last_peered":"2024-03-23T23:42:07.895117+0000","last_clean":"2024-03-23T23:42:07.895117+0000","last_became_active":"2024-03-23T23:42:07.881460+0000","last_became_peered":"2024-03-23T23:42:07.881460+0000","last_unstale":"2024-03-23T23:42:07.895117+0000","last_undegraded":"2024-03-23T23:42:07.895117+0000","last_fullsized":"2024-03-23T23:42:07.895117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:07.663519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876256+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:42:07.876256+0000","last_peered":"2024-03-23T23:42:07.876256+0000","last_clean":"2024-03-23T23:42:07.876256+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:42:07.876256+0000","last_undegraded":"2024-03-23T23:42:07.876256+0000","last_fullsized":"2024-03-23T23:42:07.876256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730445+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:42:36.730445+0000","last_peered":"2024-03-23T23:42:36.730445+0000","last_clean":"2024-03-23T23:42:36.730445+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:42:36.730445+0000","last_undegraded":"2024-03-23T23:42:36.730445+0000","last_fullsized":"2024-03-23T23:42:36.730445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.887515+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:42:07.887515+0000","last_peered":"2024-03-23T23:42:07.887515+0000","last_clean":"2024-03-23T23:42:07.887515+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:42:07.887515+0000","last_undegraded":"2024-03-23T23:42:07.887515+0000","last_fullsized":"2024-03-23T23:42:07.887515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724002+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:42:36.724002+0000","last_peered":"2024-03-23T23:42:36.724002+0000","last_clean":"2024-03-23T23:42:36.724002+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:42:36.724002+0000","last_undegraded":"2024-03-23T23:42:36.724002+0000","last_fullsized":"2024-03-23T23:42:36.724002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360584+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:42:08.360584+0000","last_peered":"2024-03-23T23:42:08.360584+0000","last_clean":"2024-03-23T23:42:08.360584+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:42:08.360584+0000","last_undegraded":"2024-03-23T23:42:08.360584+0000","last_fullsized":"2024-03-23T23:42:08.360584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758540+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:42:36.758540+0000","last_peered":"2024-03-23T23:42:36.758540+0000","last_clean":"2024-03-23T23:42:36.758540+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:42:36.758540+0000","last_undegraded":"2024-03-23T23:42:36.758540+0000","last_fullsized":"2024-03-23T23:42:36.758540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.383575+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:42:08.383575+0000","last_peered":"2024-03-23T23:42:08.383575+0000","last_clean":"2024-03-23T23:42:08.383575+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:42:08.383575+0000","last_undegraded":"2024-03-23T23:42:08.383575+0000","last_fullsized":"2024-03-23T23:42:08.383575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.687258+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:42:36.687258+0000","last_peered":"2024-03-23T23:42:36.687258+0000","last_clean":"2024-03-23T23:42:36.687258+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:42:36.687258+0000","last_undegraded":"2024-03-23T23:42:36.687258+0000","last_fullsized":"2024-03-23T23:42:36.687258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.671843+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:42:08.671843+0000","last_peered":"2024-03-23T23:42:08.671843+0000","last_clean":"2024-03-23T23:42:08.671843+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:42:08.671843+0000","last_undegraded":"2024-03-23T23:42:08.671843+0000","last_fullsized":"2024-03-23T23:42:08.671843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.750966+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:42:36.750966+0000","last_peered":"2024-03-23T23:42:36.750966+0000","last_clean":"2024-03-23T23:42:36.750966+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:42:36.750966+0000","last_undegraded":"2024-03-23T23:42:36.750966+0000","last_fullsized":"2024-03-23T23:42:36.750966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878413+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:42:07.878413+0000","last_peered":"2024-03-23T23:42:07.878413+0000","last_clean":"2024-03-23T23:42:07.878413+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:42:07.878413+0000","last_undegraded":"2024-03-23T23:42:07.878413+0000","last_fullsized":"2024-03-23T23:42:07.878413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751369+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:42:36.751369+0000","last_peered":"2024-03-23T23:42:36.751369+0000","last_clean":"2024-03-23T23:42:36.751369+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:42:36.751369+0000","last_undegraded":"2024-03-23T23:42:36.751369+0000","last_fullsized":"2024-03-23T23:42:36.751369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.364753+0000","last_change":"2024-03-23T23:42:08.364753+0000","last_active":"2024-03-23T23:42:08.364753+0000","last_peered":"2024-03-23T23:42:08.364753+0000","last_clean":"2024-03-23T23:42:08.364753+0000","last_became_active":"2024-03-23T23:42:08.361512+0000","last_became_peered":"2024-03-23T23:42:08.361512+0000","last_unstale":"2024-03-23T23:42:08.364753+0000","last_undegraded":"2024-03-23T23:42:08.364753+0000","last_fullsized":"2024-03-23T23:42:08.364753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:06:40.047574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729398+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:42:36.729398+0000","last_peered":"2024-03-23T23:42:36.729398+0000","last_clean":"2024-03-23T23:42:36.729398+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:42:36.729398+0000","last_undegraded":"2024-03-23T23:42:36.729398+0000","last_fullsized":"2024-03-23T23:42:36.729398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:37.374806+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:42:37.374806+0000","last_peered":"2024-03-23T23:42:37.374806+0000","last_clean":"2024-03-23T23:42:37.374806+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:42:37.374806+0000","last_undegraded":"2024-03-23T23:42:37.374806+0000","last_fullsized":"2024-03-23T23:42:37.374806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722634+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:42:36.722634+0000","last_peered":"2024-03-23T23:42:36.722634+0000","last_clean":"2024-03-23T23:42:36.722634+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:42:36.722634+0000","last_undegraded":"2024-03-23T23:42:36.722634+0000","last_fullsized":"2024-03-23T23:42:36.722634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881632+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:42:07.881632+0000","last_peered":"2024-03-23T23:42:07.881632+0000","last_clean":"2024-03-23T23:42:07.881632+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:42:07.881632+0000","last_undegraded":"2024-03-23T23:42:07.881632+0000","last_fullsized":"2024-03-23T23:42:07.881632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725181+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:42:36.725181+0000","last_peered":"2024-03-23T23:42:36.725181+0000","last_clean":"2024-03-23T23:42:36.725181+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:42:36.725181+0000","last_undegraded":"2024-03-23T23:42:36.725181+0000","last_fullsized":"2024-03-23T23:42:36.725181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878269+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:42:07.878269+0000","last_peered":"2024-03-23T23:42:07.878269+0000","last_clean":"2024-03-23T23:42:07.878269+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:42:07.878269+0000","last_undegraded":"2024-03-23T23:42:07.878269+0000","last_fullsized":"2024-03-23T23:42:07.878269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726066+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:42:36.726066+0000","last_peered":"2024-03-23T23:42:36.726066+0000","last_clean":"2024-03-23T23:42:36.726066+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:42:36.726066+0000","last_undegraded":"2024-03-23T23:42:36.726066+0000","last_fullsized":"2024-03-23T23:42:36.726066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727328+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.727328+0000","last_peered":"2024-03-23T23:42:36.727328+0000","last_clean":"2024-03-23T23:42:36.727328+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:42:36.727328+0000","last_undegraded":"2024-03-23T23:42:36.727328+0000","last_fullsized":"2024-03-23T23:42:36.727328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722748+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:42:36.722748+0000","last_peered":"2024-03-23T23:42:36.722748+0000","last_clean":"2024-03-23T23:42:36.722748+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:42:36.722748+0000","last_undegraded":"2024-03-23T23:42:36.722748+0000","last_fullsized":"2024-03-23T23:42:36.722748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.745305+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:42:36.745305+0000","last_peered":"2024-03-23T23:42:36.745305+0000","last_clean":"2024-03-23T23:42:36.745305+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:42:36.745305+0000","last_undegraded":"2024-03-23T23:42:36.745305+0000","last_fullsized":"2024-03-23T23:42:36.745305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.756558+0000","last_change":"2024-03-23T23:42:07.885385+0000","last_active":"2024-03-23T23:42:36.756558+0000","last_peered":"2024-03-23T23:42:36.756558+0000","last_clean":"2024-03-23T23:42:36.756558+0000","last_became_active":"2024-03-23T23:42:07.884976+0000","last_became_peered":"2024-03-23T23:42:07.884976+0000","last_unstale":"2024-03-23T23:42:36.756558+0000","last_undegraded":"2024-03-23T23:42:36.756558+0000","last_fullsized":"2024-03-23T23:42:36.756558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:40:36.593277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751015+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:42:36.751015+0000","last_peered":"2024-03-23T23:42:36.751015+0000","last_clean":"2024-03-23T23:42:36.751015+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:42:36.751015+0000","last_undegraded":"2024-03-23T23:42:36.751015+0000","last_fullsized":"2024-03-23T23:42:36.751015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722761+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:42:36.722761+0000","last_peered":"2024-03-23T23:42:36.722761+0000","last_clean":"2024-03-23T23:42:36.722761+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:42:36.722761+0000","last_undegraded":"2024-03-23T23:42:36.722761+0000","last_fullsized":"2024-03-23T23:42:36.722761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725812+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:42:36.725812+0000","last_peered":"2024-03-23T23:42:36.725812+0000","last_clean":"2024-03-23T23:42:36.725812+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:42:36.725812+0000","last_undegraded":"2024-03-23T23:42:36.725812+0000","last_fullsized":"2024-03-23T23:42:36.725812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722472+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:42:36.722472+0000","last_peered":"2024-03-23T23:42:36.722472+0000","last_clean":"2024-03-23T23:42:36.722472+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:42:36.722472+0000","last_undegraded":"2024-03-23T23:42:36.722472+0000","last_fullsized":"2024-03-23T23:42:36.722472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742130+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:42:36.742130+0000","last_peered":"2024-03-23T23:42:36.742130+0000","last_clean":"2024-03-23T23:42:36.742130+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:42:36.742130+0000","last_undegraded":"2024-03-23T23:42:36.742130+0000","last_fullsized":"2024-03-23T23:42:36.742130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725603+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:42:36.725603+0000","last_peered":"2024-03-23T23:42:36.725603+0000","last_clean":"2024-03-23T23:42:36.725603+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:42:36.725603+0000","last_undegraded":"2024-03-23T23:42:36.725603+0000","last_fullsized":"2024-03-23T23:42:36.725603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738526+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:42:36.738526+0000","last_peered":"2024-03-23T23:42:36.738526+0000","last_clean":"2024-03-23T23:42:36.738526+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:42:36.738526+0000","last_undegraded":"2024-03-23T23:42:36.738526+0000","last_fullsized":"2024-03-23T23:42:36.738526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748177+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:42:36.748177+0000","last_peered":"2024-03-23T23:42:36.748177+0000","last_clean":"2024-03-23T23:42:36.748177+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:42:36.748177+0000","last_undegraded":"2024-03-23T23:42:36.748177+0000","last_fullsized":"2024-03-23T23:42:36.748177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.719687+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:42:36.719687+0000","last_peered":"2024-03-23T23:42:36.719687+0000","last_clean":"2024-03-23T23:42:36.719687+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:42:36.719687+0000","last_undegraded":"2024-03-23T23:42:36.719687+0000","last_fullsized":"2024-03-23T23:42:36.719687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70286,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":864,"num_read_kb":513,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210858,"data_compressed":5031,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":456997,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1111358,"data_compressed":43227,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456927,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457305,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1050496,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456644,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":521511,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":457983,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":519955,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456469,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":457037,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1049642,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65706,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64150,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65068,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:42:44.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T23:42:44.753 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:45.039 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:44 smithi099 ceph-mon[31715]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:45.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:44 smithi149 ceph-mon[36083]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:45.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:44 smithi174 ceph-mon[37002]: from='client.24904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:46.072 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:45 smithi099 ceph-mon[31715]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:46.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:45 smithi149 ceph-mon[36083]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:46.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:45 smithi174 ceph-mon[37002]: pgmap v366: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 -- 172.21.15.99:0/1136454631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 msgr2=0x7f42bc10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1136454631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc10f080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009fc0 tx=0x7f42b400b330 comp rx=0 tx=0).stop 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 -- 172.21.15.99:0/1136454631 shutdown_connections 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1136454631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc10f080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1136454631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1136454631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 0x7f42bc106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 -- 172.21.15.99:0/1136454631 >> 172.21.15.99:0/1136454631 conn(0x7f42bc0fdc40 msgr2=0x7f42bc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 -- 172.21.15.99:0/1136454631 shutdown_connections 2024-03-23T23:42:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.888+0000 7f42c0c00700 1 -- 172.21.15.99:0/1136454631 wait complete. 2024-03-23T23:42:46.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 Processor -- start 2024-03-23T23:42:46.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 -- start start 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 0x7f42bc1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1124f0 con 0x7f42bc1074b0 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc112370 con 0x7f42bc102d80 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.890+0000 7f42c0c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc112670 con 0x7f42bc1023d0 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57978/0 (socket says 172.21.15.99:57978) 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 -- 172.21.15.99:0/1753086893 learned_addr learned my addr 172.21.15.99:0/1753086893 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9400700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 0x7f42bc1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42ba800700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:46.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 -- 172.21.15.99:0/1753086893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 msgr2=0x7f42bc1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:46.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 0x7f42bc1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:46.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 -- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 msgr2=0x7f42bc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:46.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:46.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 -- 172.21.15.99:0/1753086893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc19b130 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.891+0000 7f42b9e00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ec10 tx=0x7f42ac00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.892+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00cd40 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.892+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42ac004500 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.892+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac010690 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.892+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc19b420 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.892+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc1a76e0 con 0x7f42bc1023d0 2024-03-23T23:42:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.893+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc068e50 con 0x7f42bc1023d0 2024-03-23T23:42:46.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.896+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f42ac003fa0 con 0x7f42bc1023d0 2024-03-23T23:42:46.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.898+0000 7f42b2a00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 0x7f42a00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:46.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.898+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f42ac0cede0 con 0x7f42bc1023d0 2024-03-23T23:42:46.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.898+0000 7f42b9400700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 0x7f42a00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:46.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.898+0000 7f42b9400700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 0x7f42a00a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009680 tx=0x7f42a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:46.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:46.906+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42ac09be00 con 0x7f42bc1023d0 2024-03-23T23:42:47.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.137+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42bc0ff570 con 0x7f42a00a2910 2024-03-23T23:42:47.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.149+0000 7f42b2a00700 1 -- 172.21.15.99:0/1753086893 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259182 (secure 0 0 0) 0x7f42bc0ff570 con 0x7f42a00a2910 2024-03-23T23:42:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.154+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 msgr2=0x7f42a00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.154+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 0x7f42a00a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009680 tx=0x7f42a8009260 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.154+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 msgr2=0x7f42bc1a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.154+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ec10 tx=0x7f42ac00ef20 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 shutdown_connections 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f42a00a2910 0x7f42a00a4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42bc1074b0 0x7f42bc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc1023d0 0x7f42bc1a0810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 --2- 172.21.15.99:0/1753086893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42bc102d80 0x7f42bc1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 >> 172.21.15.99:0/1753086893 conn(0x7f42bc0fdc40 msgr2=0x7f42bc10d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 shutdown_connections 2024-03-23T23:42:47.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:47.155+0000 7f42c0c00700 1 -- 172.21.15.99:0/1753086893 wait complete. 2024-03-23T23:42:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:42:47.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:47 smithi099 ceph-mon[31715]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:48.093 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-03-23T23:42:46.759648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660888,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1034,"num_read_kb":641,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381664,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560160,"statfs":{"total":1151940427776,"available":1151549603840,"internally_reserved":0,"allocated":11993088,"data_stored":7452324,"data_compressed":181782,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":67,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003875"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751476+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:42:36.751476+0000","last_peered":"2024-03-23T23:42:36.751476+0000","last_clean":"2024-03-23T23:42:36.751476+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:42:36.751476+0000","last_undegraded":"2024-03-23T23:42:36.751476+0000","last_fullsized":"2024-03-23T23:42:36.751476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758307+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:42:36.758307+0000","last_peered":"2024-03-23T23:42:36.758307+0000","last_clean":"2024-03-23T23:42:36.758307+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:42:36.758307+0000","last_undegraded":"2024-03-23T23:42:36.758307+0000","last_fullsized":"2024-03-23T23:42:36.758307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761253+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:42:36.761253+0000","last_peered":"2024-03-23T23:42:36.761253+0000","last_clean":"2024-03-23T23:42:36.761253+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:42:36.761253+0000","last_undegraded":"2024-03-23T23:42:36.761253+0000","last_fullsized":"2024-03-23T23:42:36.761253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722922+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:42:36.722922+0000","last_peered":"2024-03-23T23:42:36.722922+0000","last_clean":"2024-03-23T23:42:36.722922+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:42:36.722922+0000","last_undegraded":"2024-03-23T23:42:36.722922+0000","last_fullsized":"2024-03-23T23:42:36.722922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724866+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:42:36.724866+0000","last_peered":"2024-03-23T23:42:36.724866+0000","last_clean":"2024-03-23T23:42:36.724866+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:42:36.724866+0000","last_undegraded":"2024-03-23T23:42:36.724866+0000","last_fullsized":"2024-03-23T23:42:36.724866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724820+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:42:36.724820+0000","last_peered":"2024-03-23T23:42:36.724820+0000","last_clean":"2024-03-23T23:42:36.724820+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:42:36.724820+0000","last_undegraded":"2024-03-23T23:42:36.724820+0000","last_fullsized":"2024-03-23T23:42:36.724820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.747830+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:42:36.747830+0000","last_peered":"2024-03-23T23:42:36.747830+0000","last_clean":"2024-03-23T23:42:36.747830+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:42:36.747830+0000","last_undegraded":"2024-03-23T23:42:36.747830+0000","last_fullsized":"2024-03-23T23:42:36.747830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738623+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:42:36.738623+0000","last_peered":"2024-03-23T23:42:36.738623+0000","last_clean":"2024-03-23T23:42:36.738623+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:42:36.738623+0000","last_undegraded":"2024-03-23T23:42:36.738623+0000","last_fullsized":"2024-03-23T23:42:36.738623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728376+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:42:36.728376+0000","last_peered":"2024-03-23T23:42:36.728376+0000","last_clean":"2024-03-23T23:42:36.728376+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:42:36.728376+0000","last_undegraded":"2024-03-23T23:42:36.728376+0000","last_fullsized":"2024-03-23T23:42:36.728376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.723262+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:42:36.723262+0000","last_peered":"2024-03-23T23:42:36.723262+0000","last_clean":"2024-03-23T23:42:36.723262+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:42:36.723262+0000","last_undegraded":"2024-03-23T23:42:36.723262+0000","last_fullsized":"2024-03-23T23:42:36.723262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.720783+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:42:36.720783+0000","last_peered":"2024-03-23T23:42:36.720783+0000","last_clean":"2024-03-23T23:42:36.720783+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:42:36.720783+0000","last_undegraded":"2024-03-23T23:42:36.720783+0000","last_fullsized":"2024-03-23T23:42:36.720783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.715456+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:42:36.715456+0000","last_peered":"2024-03-23T23:42:36.715456+0000","last_clean":"2024-03-23T23:42:36.715456+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:42:36.715456+0000","last_undegraded":"2024-03-23T23:42:36.715456+0000","last_fullsized":"2024-03-23T23:42:36.715456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.746606+0000","last_change":"2024-03-23T23:42:07.892099+0000","last_active":"2024-03-23T23:42:36.746606+0000","last_peered":"2024-03-23T23:42:36.746606+0000","last_clean":"2024-03-23T23:42:36.746606+0000","last_became_active":"2024-03-23T23:42:07.891702+0000","last_became_peered":"2024-03-23T23:42:07.891702+0000","last_unstale":"2024-03-23T23:42:36.746606+0000","last_undegraded":"2024-03-23T23:42:36.746606+0000","last_fullsized":"2024-03-23T23:42:36.746606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:52:44.779482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.759908+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:42:36.759908+0000","last_peered":"2024-03-23T23:42:36.759908+0000","last_clean":"2024-03-23T23:42:36.759908+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:42:36.759908+0000","last_undegraded":"2024-03-23T23:42:36.759908+0000","last_fullsized":"2024-03-23T23:42:36.759908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758585+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:42:36.758585+0000","last_peered":"2024-03-23T23:42:36.758585+0000","last_clean":"2024-03-23T23:42:36.758585+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:42:36.758585+0000","last_undegraded":"2024-03-23T23:42:36.758585+0000","last_fullsized":"2024-03-23T23:42:36.758585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758550+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:42:36.758550+0000","last_peered":"2024-03-23T23:42:36.758550+0000","last_clean":"2024-03-23T23:42:36.758550+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:42:36.758550+0000","last_undegraded":"2024-03-23T23:42:36.758550+0000","last_fullsized":"2024-03-23T23:42:36.758550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751597+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:42:36.751597+0000","last_peered":"2024-03-23T23:42:36.751597+0000","last_clean":"2024-03-23T23:42:36.751597+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:42:36.751597+0000","last_undegraded":"2024-03-23T23:42:36.751597+0000","last_fullsized":"2024-03-23T23:42:36.751597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749275+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:42:36.749275+0000","last_peered":"2024-03-23T23:42:36.749275+0000","last_clean":"2024-03-23T23:42:36.749275+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:42:36.749275+0000","last_undegraded":"2024-03-23T23:42:36.749275+0000","last_fullsized":"2024-03-23T23:42:36.749275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.721704+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:42:36.721704+0000","last_peered":"2024-03-23T23:42:36.721704+0000","last_clean":"2024-03-23T23:42:36.721704+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:42:36.721704+0000","last_undegraded":"2024-03-23T23:42:36.721704+0000","last_fullsized":"2024-03-23T23:42:36.721704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:41.403708+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:42:41.403708+0000","last_peered":"2024-03-23T23:42:41.403708+0000","last_clean":"2024-03-23T23:42:41.403708+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:42:41.403708+0000","last_undegraded":"2024-03-23T23:42:41.403708+0000","last_fullsized":"2024-03-23T23:42:41.403708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740593+0000","last_change":"2024-03-23T23:42:07.890419+0000","last_active":"2024-03-23T23:42:36.740593+0000","last_peered":"2024-03-23T23:42:36.740593+0000","last_clean":"2024-03-23T23:42:36.740593+0000","last_became_active":"2024-03-23T23:42:07.889998+0000","last_became_peered":"2024-03-23T23:42:07.889998+0000","last_unstale":"2024-03-23T23:42:36.740593+0000","last_undegraded":"2024-03-23T23:42:36.740593+0000","last_fullsized":"2024-03-23T23:42:36.740593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:35:13.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.672737+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:42:08.672737+0000","last_peered":"2024-03-23T23:42:08.672737+0000","last_clean":"2024-03-23T23:42:08.672737+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:42:08.672737+0000","last_undegraded":"2024-03-23T23:42:08.672737+0000","last_fullsized":"2024-03-23T23:42:08.672737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761400+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:42:36.761400+0000","last_peered":"2024-03-23T23:42:36.761400+0000","last_clean":"2024-03-23T23:42:36.761400+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:42:36.761400+0000","last_undegraded":"2024-03-23T23:42:36.761400+0000","last_fullsized":"2024-03-23T23:42:36.761400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.892818+0000","last_change":"2024-03-23T23:42:07.892818+0000","last_active":"2024-03-23T23:42:07.892818+0000","last_peered":"2024-03-23T23:42:07.892818+0000","last_clean":"2024-03-23T23:42:07.892818+0000","last_became_active":"2024-03-23T23:42:07.880207+0000","last_became_peered":"2024-03-23T23:42:07.880207+0000","last_unstale":"2024-03-23T23:42:07.892818+0000","last_undegraded":"2024-03-23T23:42:07.892818+0000","last_fullsized":"2024-03-23T23:42:07.892818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:04:54.088457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743533+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:42:36.743533+0000","last_peered":"2024-03-23T23:42:36.743533+0000","last_clean":"2024-03-23T23:42:36.743533+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:42:36.743533+0000","last_undegraded":"2024-03-23T23:42:36.743533+0000","last_fullsized":"2024-03-23T23:42:36.743533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876259+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:42:07.876259+0000","last_peered":"2024-03-23T23:42:07.876259+0000","last_clean":"2024-03-23T23:42:07.876259+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:42:07.876259+0000","last_undegraded":"2024-03-23T23:42:07.876259+0000","last_fullsized":"2024-03-23T23:42:07.876259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727324+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:42:36.727324+0000","last_peered":"2024-03-23T23:42:36.727324+0000","last_clean":"2024-03-23T23:42:36.727324+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:42:36.727324+0000","last_undegraded":"2024-03-23T23:42:36.727324+0000","last_fullsized":"2024-03-23T23:42:36.727324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":126,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:41.415412+0000","last_change":"2024-03-23T23:42:08.388477+0000","last_active":"2024-03-23T23:42:41.415412+0000","last_peered":"2024-03-23T23:42:41.415412+0000","last_clean":"2024-03-23T23:42:41.415412+0000","last_became_active":"2024-03-23T23:42:08.384694+0000","last_became_peered":"2024-03-23T23:42:08.384694+0000","last_unstale":"2024-03-23T23:42:41.415412+0000","last_undegraded":"2024-03-23T23:42:41.415412+0000","last_fullsized":"2024-03-23T23:42:41.415412+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:13.778147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.718444+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.718444+0000","last_peered":"2024-03-23T23:42:36.718444+0000","last_clean":"2024-03-23T23:42:36.718444+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:42:36.718444+0000","last_undegraded":"2024-03-23T23:42:36.718444+0000","last_fullsized":"2024-03-23T23:42:36.718444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.882733+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:42:07.882733+0000","last_peered":"2024-03-23T23:42:07.882733+0000","last_clean":"2024-03-23T23:42:07.882733+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:42:07.882733+0000","last_undegraded":"2024-03-23T23:42:07.882733+0000","last_fullsized":"2024-03-23T23:42:07.882733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743447+0000","last_change":"2024-03-23T23:42:08.673791+0000","last_active":"2024-03-23T23:42:36.743447+0000","last_peered":"2024-03-23T23:42:36.743447+0000","last_clean":"2024-03-23T23:42:36.743447+0000","last_became_active":"2024-03-23T23:42:08.673494+0000","last_became_peered":"2024-03-23T23:42:08.673494+0000","last_unstale":"2024-03-23T23:42:36.743447+0000","last_undegraded":"2024-03-23T23:42:36.743447+0000","last_fullsized":"2024-03-23T23:42:36.743447+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:22:10.209266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.877485+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:42:07.877485+0000","last_peered":"2024-03-23T23:42:07.877485+0000","last_clean":"2024-03-23T23:42:07.877485+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:42:07.877485+0000","last_undegraded":"2024-03-23T23:42:07.877485+0000","last_fullsized":"2024-03-23T23:42:07.877485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738460+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:42:36.738460+0000","last_peered":"2024-03-23T23:42:36.738460+0000","last_clean":"2024-03-23T23:42:36.738460+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:42:36.738460+0000","last_undegraded":"2024-03-23T23:42:36.738460+0000","last_fullsized":"2024-03-23T23:42:36.738460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.880586+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:42:07.880586+0000","last_peered":"2024-03-23T23:42:07.880586+0000","last_clean":"2024-03-23T23:42:07.880586+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:42:07.880586+0000","last_undegraded":"2024-03-23T23:42:07.880586+0000","last_fullsized":"2024-03-23T23:42:07.880586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749958+0000","last_change":"2024-03-23T23:42:07.891565+0000","last_active":"2024-03-23T23:42:36.749958+0000","last_peered":"2024-03-23T23:42:36.749958+0000","last_clean":"2024-03-23T23:42:36.749958+0000","last_became_active":"2024-03-23T23:42:07.890328+0000","last_became_peered":"2024-03-23T23:42:07.890328+0000","last_unstale":"2024-03-23T23:42:36.749958+0000","last_undegraded":"2024-03-23T23:42:36.749958+0000","last_fullsized":"2024-03-23T23:42:36.749958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:44:07.467469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":201,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:45.382347+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:42:45.382347+0000","last_peered":"2024-03-23T23:42:45.382347+0000","last_clean":"2024-03-23T23:42:45.382347+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:42:45.382347+0000","last_undegraded":"2024-03-23T23:42:45.382347+0000","last_fullsized":"2024-03-23T23:42:45.382347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":177,"num_read_kb":177,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742341+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:42:36.742341+0000","last_peered":"2024-03-23T23:42:36.742341+0000","last_clean":"2024-03-23T23:42:36.742341+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:42:36.742341+0000","last_undegraded":"2024-03-23T23:42:36.742341+0000","last_fullsized":"2024-03-23T23:42:36.742341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878070+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:42:07.878070+0000","last_peered":"2024-03-23T23:42:07.878070+0000","last_clean":"2024-03-23T23:42:07.878070+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:42:07.878070+0000","last_undegraded":"2024-03-23T23:42:07.878070+0000","last_fullsized":"2024-03-23T23:42:07.878070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760716+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:42:36.760716+0000","last_peered":"2024-03-23T23:42:36.760716+0000","last_clean":"2024-03-23T23:42:36.760716+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:42:36.760716+0000","last_undegraded":"2024-03-23T23:42:36.760716+0000","last_fullsized":"2024-03-23T23:42:36.760716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881789+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:42:07.881789+0000","last_peered":"2024-03-23T23:42:07.881789+0000","last_clean":"2024-03-23T23:42:07.881789+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:42:07.881789+0000","last_undegraded":"2024-03-23T23:42:07.881789+0000","last_fullsized":"2024-03-23T23:42:07.881789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728388+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:42:36.728388+0000","last_peered":"2024-03-23T23:42:36.728388+0000","last_clean":"2024-03-23T23:42:36.728388+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:42:36.728388+0000","last_undegraded":"2024-03-23T23:42:36.728388+0000","last_fullsized":"2024-03-23T23:42:36.728388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":240,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:45.417901+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:42:45.417901+0000","last_peered":"2024-03-23T23:42:45.417901+0000","last_clean":"2024-03-23T23:42:45.417901+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:42:45.417901+0000","last_undegraded":"2024-03-23T23:42:45.417901+0000","last_fullsized":"2024-03-23T23:42:45.417901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62800,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":154,"num_write":38,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725831+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:42:36.725831+0000","last_peered":"2024-03-23T23:42:36.725831+0000","last_clean":"2024-03-23T23:42:36.725831+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:42:36.725831+0000","last_undegraded":"2024-03-23T23:42:36.725831+0000","last_fullsized":"2024-03-23T23:42:36.725831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878290+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:42:07.878290+0000","last_peered":"2024-03-23T23:42:07.878290+0000","last_clean":"2024-03-23T23:42:07.878290+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:42:07.878290+0000","last_undegraded":"2024-03-23T23:42:07.878290+0000","last_fullsized":"2024-03-23T23:42:07.878290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740229+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:42:36.740229+0000","last_peered":"2024-03-23T23:42:36.740229+0000","last_clean":"2024-03-23T23:42:36.740229+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:42:36.740229+0000","last_undegraded":"2024-03-23T23:42:36.740229+0000","last_fullsized":"2024-03-23T23:42:36.740229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881072+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:42:07.881072+0000","last_peered":"2024-03-23T23:42:07.881072+0000","last_clean":"2024-03-23T23:42:07.881072+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:42:07.881072+0000","last_undegraded":"2024-03-23T23:42:07.881072+0000","last_fullsized":"2024-03-23T23:42:07.881072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.736988+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:42:36.736988+0000","last_peered":"2024-03-23T23:42:36.736988+0000","last_clean":"2024-03-23T23:42:36.736988+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:42:36.736988+0000","last_undegraded":"2024-03-23T23:42:36.736988+0000","last_fullsized":"2024-03-23T23:42:36.736988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":116,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:42.407346+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:42:42.407346+0000","last_peered":"2024-03-23T23:42:42.407346+0000","last_clean":"2024-03-23T23:42:42.407346+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:42:42.407346+0000","last_undegraded":"2024-03-23T23:42:42.407346+0000","last_fullsized":"2024-03-23T23:42:42.407346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725306+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:42:36.725306+0000","last_peered":"2024-03-23T23:42:36.725306+0000","last_clean":"2024-03-23T23:42:36.725306+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:42:36.725306+0000","last_undegraded":"2024-03-23T23:42:36.725306+0000","last_fullsized":"2024-03-23T23:42:36.725306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.875951+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:42:07.875951+0000","last_peered":"2024-03-23T23:42:07.875951+0000","last_clean":"2024-03-23T23:42:07.875951+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:42:07.875951+0000","last_undegraded":"2024-03-23T23:42:07.875951+0000","last_fullsized":"2024-03-23T23:42:07.875951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726634+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:42:36.726634+0000","last_peered":"2024-03-23T23:42:36.726634+0000","last_clean":"2024-03-23T23:42:36.726634+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:42:36.726634+0000","last_undegraded":"2024-03-23T23:42:36.726634+0000","last_fullsized":"2024-03-23T23:42:36.726634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876451+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:42:07.876451+0000","last_peered":"2024-03-23T23:42:07.876451+0000","last_clean":"2024-03-23T23:42:07.876451+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:42:07.876451+0000","last_undegraded":"2024-03-23T23:42:07.876451+0000","last_fullsized":"2024-03-23T23:42:07.876451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722437+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:42:36.722437+0000","last_peered":"2024-03-23T23:42:36.722437+0000","last_clean":"2024-03-23T23:42:36.722437+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:42:36.722437+0000","last_undegraded":"2024-03-23T23:42:36.722437+0000","last_fullsized":"2024-03-23T23:42:36.722437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.888190+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:42:07.888190+0000","last_peered":"2024-03-23T23:42:07.888190+0000","last_clean":"2024-03-23T23:42:07.888190+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:42:07.888190+0000","last_undegraded":"2024-03-23T23:42:07.888190+0000","last_fullsized":"2024-03-23T23:42:07.888190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728326+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:42:36.728326+0000","last_peered":"2024-03-23T23:42:36.728326+0000","last_clean":"2024-03-23T23:42:36.728326+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:42:36.728326+0000","last_undegraded":"2024-03-23T23:42:36.728326+0000","last_fullsized":"2024-03-23T23:42:36.728326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:43.406645+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:42:43.406645+0000","last_peered":"2024-03-23T23:42:43.406645+0000","last_clean":"2024-03-23T23:42:43.406645+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:42:43.406645+0000","last_undegraded":"2024-03-23T23:42:43.406645+0000","last_fullsized":"2024-03-23T23:42:43.406645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.757668+0000","last_change":"2024-03-23T23:42:07.890902+0000","last_active":"2024-03-23T23:42:36.757668+0000","last_peered":"2024-03-23T23:42:36.757668+0000","last_clean":"2024-03-23T23:42:36.757668+0000","last_became_active":"2024-03-23T23:42:07.890514+0000","last_became_peered":"2024-03-23T23:42:07.890514+0000","last_unstale":"2024-03-23T23:42:36.757668+0000","last_undegraded":"2024-03-23T23:42:36.757668+0000","last_fullsized":"2024-03-23T23:42:36.757668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:42.343732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725916+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:42:36.725916+0000","last_peered":"2024-03-23T23:42:36.725916+0000","last_clean":"2024-03-23T23:42:36.725916+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:42:36.725916+0000","last_undegraded":"2024-03-23T23:42:36.725916+0000","last_fullsized":"2024-03-23T23:42:36.725916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:40.404140+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:42:40.404140+0000","last_peered":"2024-03-23T23:42:40.404140+0000","last_clean":"2024-03-23T23:42:40.404140+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:42:40.404140+0000","last_undegraded":"2024-03-23T23:42:40.404140+0000","last_fullsized":"2024-03-23T23:42:40.404140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726081+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:42:36.726081+0000","last_peered":"2024-03-23T23:42:36.726081+0000","last_clean":"2024-03-23T23:42:36.726081+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:42:36.726081+0000","last_undegraded":"2024-03-23T23:42:36.726081+0000","last_fullsized":"2024-03-23T23:42:36.726081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360541+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:42:08.360541+0000","last_peered":"2024-03-23T23:42:08.360541+0000","last_clean":"2024-03-23T23:42:08.360541+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:42:08.360541+0000","last_undegraded":"2024-03-23T23:42:08.360541+0000","last_fullsized":"2024-03-23T23:42:08.360541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878026+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:42:07.878026+0000","last_peered":"2024-03-23T23:42:07.878026+0000","last_clean":"2024-03-23T23:42:07.878026+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:42:07.878026+0000","last_undegraded":"2024-03-23T23:42:07.878026+0000","last_fullsized":"2024-03-23T23:42:07.878026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729760+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:42:36.729760+0000","last_peered":"2024-03-23T23:42:36.729760+0000","last_clean":"2024-03-23T23:42:36.729760+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:42:36.729760+0000","last_undegraded":"2024-03-23T23:42:36.729760+0000","last_fullsized":"2024-03-23T23:42:36.729760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.895117+0000","last_change":"2024-03-23T23:42:07.895117+0000","last_active":"2024-03-23T23:42:07.895117+0000","last_peered":"2024-03-23T23:42:07.895117+0000","last_clean":"2024-03-23T23:42:07.895117+0000","last_became_active":"2024-03-23T23:42:07.881460+0000","last_became_peered":"2024-03-23T23:42:07.881460+0000","last_unstale":"2024-03-23T23:42:07.895117+0000","last_undegraded":"2024-03-23T23:42:07.895117+0000","last_fullsized":"2024-03-23T23:42:07.895117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:07.663519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876256+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:42:07.876256+0000","last_peered":"2024-03-23T23:42:07.876256+0000","last_clean":"2024-03-23T23:42:07.876256+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:42:07.876256+0000","last_undegraded":"2024-03-23T23:42:07.876256+0000","last_fullsized":"2024-03-23T23:42:07.876256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730445+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:42:36.730445+0000","last_peered":"2024-03-23T23:42:36.730445+0000","last_clean":"2024-03-23T23:42:36.730445+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:42:36.730445+0000","last_undegraded":"2024-03-23T23:42:36.730445+0000","last_fullsized":"2024-03-23T23:42:36.730445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.887515+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:42:07.887515+0000","last_peered":"2024-03-23T23:42:07.887515+0000","last_clean":"2024-03-23T23:42:07.887515+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:42:07.887515+0000","last_undegraded":"2024-03-23T23:42:07.887515+0000","last_fullsized":"2024-03-23T23:42:07.887515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724002+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:42:36.724002+0000","last_peered":"2024-03-23T23:42:36.724002+0000","last_clean":"2024-03-23T23:42:36.724002+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:42:36.724002+0000","last_undegraded":"2024-03-23T23:42:36.724002+0000","last_fullsized":"2024-03-23T23:42:36.724002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360584+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:42:08.360584+0000","last_peered":"2024-03-23T23:42:08.360584+0000","last_clean":"2024-03-23T23:42:08.360584+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:42:08.360584+0000","last_undegraded":"2024-03-23T23:42:08.360584+0000","last_fullsized":"2024-03-23T23:42:08.360584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758540+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:42:36.758540+0000","last_peered":"2024-03-23T23:42:36.758540+0000","last_clean":"2024-03-23T23:42:36.758540+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:42:36.758540+0000","last_undegraded":"2024-03-23T23:42:36.758540+0000","last_fullsized":"2024-03-23T23:42:36.758540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.383575+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:42:08.383575+0000","last_peered":"2024-03-23T23:42:08.383575+0000","last_clean":"2024-03-23T23:42:08.383575+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:42:08.383575+0000","last_undegraded":"2024-03-23T23:42:08.383575+0000","last_fullsized":"2024-03-23T23:42:08.383575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.687258+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:42:36.687258+0000","last_peered":"2024-03-23T23:42:36.687258+0000","last_clean":"2024-03-23T23:42:36.687258+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:42:36.687258+0000","last_undegraded":"2024-03-23T23:42:36.687258+0000","last_fullsized":"2024-03-23T23:42:36.687258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.671843+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:42:08.671843+0000","last_peered":"2024-03-23T23:42:08.671843+0000","last_clean":"2024-03-23T23:42:08.671843+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:42:08.671843+0000","last_undegraded":"2024-03-23T23:42:08.671843+0000","last_fullsized":"2024-03-23T23:42:08.671843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.750966+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:42:36.750966+0000","last_peered":"2024-03-23T23:42:36.750966+0000","last_clean":"2024-03-23T23:42:36.750966+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:42:36.750966+0000","last_undegraded":"2024-03-23T23:42:36.750966+0000","last_fullsized":"2024-03-23T23:42:36.750966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878413+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:42:07.878413+0000","last_peered":"2024-03-23T23:42:07.878413+0000","last_clean":"2024-03-23T23:42:07.878413+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:42:07.878413+0000","last_undegraded":"2024-03-23T23:42:07.878413+0000","last_fullsized":"2024-03-23T23:42:07.878413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751369+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:42:36.751369+0000","last_peered":"2024-03-23T23:42:36.751369+0000","last_clean":"2024-03-23T23:42:36.751369+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:42:36.751369+0000","last_undegraded":"2024-03-23T23:42:36.751369+0000","last_fullsized":"2024-03-23T23:42:36.751369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.364753+0000","last_change":"2024-03-23T23:42:08.364753+0000","last_active":"2024-03-23T23:42:08.364753+0000","last_peered":"2024-03-23T23:42:08.364753+0000","last_clean":"2024-03-23T23:42:08.364753+0000","last_became_active":"2024-03-23T23:42:08.361512+0000","last_became_peered":"2024-03-23T23:42:08.361512+0000","last_unstale":"2024-03-23T23:42:08.364753+0000","last_undegraded":"2024-03-23T23:42:08.364753+0000","last_fullsized":"2024-03-23T23:42:08.364753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:06:40.047574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729398+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:42:36.729398+0000","last_peered":"2024-03-23T23:42:36.729398+0000","last_clean":"2024-03-23T23:42:36.729398+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:42:36.729398+0000","last_undegraded":"2024-03-23T23:42:36.729398+0000","last_fullsized":"2024-03-23T23:42:36.729398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:42.379808+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:42:42.379808+0000","last_peered":"2024-03-23T23:42:42.379808+0000","last_clean":"2024-03-23T23:42:42.379808+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:42:42.379808+0000","last_undegraded":"2024-03-23T23:42:42.379808+0000","last_fullsized":"2024-03-23T23:42:42.379808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722634+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:42:36.722634+0000","last_peered":"2024-03-23T23:42:36.722634+0000","last_clean":"2024-03-23T23:42:36.722634+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:42:36.722634+0000","last_undegraded":"2024-03-23T23:42:36.722634+0000","last_fullsized":"2024-03-23T23:42:36.722634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881632+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:42:07.881632+0000","last_peered":"2024-03-23T23:42:07.881632+0000","last_clean":"2024-03-23T23:42:07.881632+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:42:07.881632+0000","last_undegraded":"2024-03-23T23:42:07.881632+0000","last_fullsized":"2024-03-23T23:42:07.881632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725181+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:42:36.725181+0000","last_peered":"2024-03-23T23:42:36.725181+0000","last_clean":"2024-03-23T23:42:36.725181+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:42:36.725181+0000","last_undegraded":"2024-03-23T23:42:36.725181+0000","last_fullsized":"2024-03-23T23:42:36.725181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878269+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:42:07.878269+0000","last_peered":"2024-03-23T23:42:07.878269+0000","last_clean":"2024-03-23T23:42:07.878269+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:42:07.878269+0000","last_undegraded":"2024-03-23T23:42:07.878269+0000","last_fullsized":"2024-03-23T23:42:07.878269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726066+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:42:36.726066+0000","last_peered":"2024-03-23T23:42:36.726066+0000","last_clean":"2024-03-23T23:42:36.726066+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:42:36.726066+0000","last_undegraded":"2024-03-23T23:42:36.726066+0000","last_fullsized":"2024-03-23T23:42:36.726066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727328+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.727328+0000","last_peered":"2024-03-23T23:42:36.727328+0000","last_clean":"2024-03-23T23:42:36.727328+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:42:36.727328+0000","last_undegraded":"2024-03-23T23:42:36.727328+0000","last_fullsized":"2024-03-23T23:42:36.727328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722748+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:42:36.722748+0000","last_peered":"2024-03-23T23:42:36.722748+0000","last_clean":"2024-03-23T23:42:36.722748+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:42:36.722748+0000","last_undegraded":"2024-03-23T23:42:36.722748+0000","last_fullsized":"2024-03-23T23:42:36.722748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.745305+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:42:36.745305+0000","last_peered":"2024-03-23T23:42:36.745305+0000","last_clean":"2024-03-23T23:42:36.745305+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:42:36.745305+0000","last_undegraded":"2024-03-23T23:42:36.745305+0000","last_fullsized":"2024-03-23T23:42:36.745305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.756558+0000","last_change":"2024-03-23T23:42:07.885385+0000","last_active":"2024-03-23T23:42:36.756558+0000","last_peered":"2024-03-23T23:42:36.756558+0000","last_clean":"2024-03-23T23:42:36.756558+0000","last_became_active":"2024-03-23T23:42:07.884976+0000","last_became_peered":"2024-03-23T23:42:07.884976+0000","last_unstale":"2024-03-23T23:42:36.756558+0000","last_undegraded":"2024-03-23T23:42:36.756558+0000","last_fullsized":"2024-03-23T23:42:36.756558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:40:36.593277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751015+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:42:36.751015+0000","last_peered":"2024-03-23T23:42:36.751015+0000","last_clean":"2024-03-23T23:42:36.751015+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:42:36.751015+0000","last_undegraded":"2024-03-23T23:42:36.751015+0000","last_fullsized":"2024-03-23T23:42:36.751015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722761+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:42:36.722761+0000","last_peered":"2024-03-23T23:42:36.722761+0000","last_clean":"2024-03-23T23:42:36.722761+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:42:36.722761+0000","last_undegraded":"2024-03-23T23:42:36.722761+0000","last_fullsized":"2024-03-23T23:42:36.722761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725812+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:42:36.725812+0000","last_peered":"2024-03-23T23:42:36.725812+0000","last_clean":"2024-03-23T23:42:36.725812+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:42:36.725812+0000","last_undegraded":"2024-03-23T23:42:36.725812+0000","last_fullsized":"2024-03-23T23:42:36.725812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722472+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:42:36.722472+0000","last_peered":"2024-03-23T23:42:36.722472+0000","last_clean":"2024-03-23T23:42:36.722472+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:42:36.722472+0000","last_undegraded":"2024-03-23T23:42:36.722472+0000","last_fullsized":"2024-03-23T23:42:36.722472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742130+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:42:36.742130+0000","last_peered":"2024-03-23T23:42:36.742130+0000","last_clean":"2024-03-23T23:42:36.742130+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:42:36.742130+0000","last_undegraded":"2024-03-23T23:42:36.742130+0000","last_fullsized":"2024-03-23T23:42:36.742130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725603+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:42:36.725603+0000","last_peered":"2024-03-23T23:42:36.725603+0000","last_clean":"2024-03-23T23:42:36.725603+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:42:36.725603+0000","last_undegraded":"2024-03-23T23:42:36.725603+0000","last_fullsized":"2024-03-23T23:42:36.725603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738526+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:42:36.738526+0000","last_peered":"2024-03-23T23:42:36.738526+0000","last_clean":"2024-03-23T23:42:36.738526+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:42:36.738526+0000","last_undegraded":"2024-03-23T23:42:36.738526+0000","last_fullsized":"2024-03-23T23:42:36.738526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748177+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:42:36.748177+0000","last_peered":"2024-03-23T23:42:36.748177+0000","last_clean":"2024-03-23T23:42:36.748177+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:42:36.748177+0000","last_undegraded":"2024-03-23T23:42:36.748177+0000","last_fullsized":"2024-03-23T23:42:36.748177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.719687+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:42:36.719687+0000","last_peered":"2024-03-23T23:42:36.719687+0000","last_clean":"2024-03-23T23:42:36.719687+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:42:36.719687+0000","last_undegraded":"2024-03-23T23:42:36.719687+0000","last_fullsized":"2024-03-23T23:42:36.719687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70286,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":908,"num_read_kb":532,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210858,"data_compressed":5031,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":456997,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1111358,"data_compressed":43227,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456927,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457305,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1050496,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456644,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":521511,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":457983,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":519955,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456469,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":457037,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1049642,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65706,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64150,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65068,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:42:48.097 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-23T23:42:48.097 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T23:42:48.097 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-23T23:42:48.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-23T23:42:48.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:47 smithi149 ceph-mon[36083]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:47 smithi174 ceph-mon[37002]: pgmap v367: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:42:48.596 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:42:48.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:48 smithi099 ceph-mon[31715]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:49.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:48 smithi149 ceph-mon[36083]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:49.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:48 smithi174 ceph-mon[37002]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:42:50.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:49 smithi099 ceph-mon[31715]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:50.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:49 smithi149 ceph-mon[36083]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:50.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:49 smithi174 ceph-mon[37002]: pgmap v368: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-23T23:42:50.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 -- 172.21.15.99:0/1269581153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc8102210 msgr2=0x7f0bc8102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:50.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1269581153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc8102210 0x7f0bc8102690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb4009ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 -- 172.21.15.99:0/1269581153 shutdown_connections 2024-03-23T23:42:50.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1269581153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc810f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1269581153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc81082f0 0x7f0bc81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1269581153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc8102210 0x7f0bc8102690 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.947+0000 7f0bd0600700 1 -- 172.21.15.99:0/1269581153 >> 172.21.15.99:0/1269581153 conn(0x7f0bc8079250 msgr2=0x7f0bc8079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:50.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.948+0000 7f0bd0600700 1 -- 172.21.15.99:0/1269581153 shutdown_connections 2024-03-23T23:42:50.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.948+0000 7f0bd0600700 1 -- 172.21.15.99:0/1269581153 wait complete. 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.949+0000 7f0bd0600700 1 Processor -- start 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.949+0000 7f0bd0600700 1 -- start start 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.949+0000 7f0bd0600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc8102210 0x7f0bc81a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.949+0000 7f0bd0600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.949+0000 7f0bd0600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 0x7f0bc819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bd0600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc81124a0 con 0x7f0bc8102210 2024-03-23T23:42:50.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bd0600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8112320 con 0x7f0bc81082f0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bd0600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8112620 con 0x7f0bc8102bd0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcdc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc8102210 0x7f0bc81a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39372/0 (socket says 172.21.15.99:39372) 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 -- 172.21.15.99:0/1260212701 learned_addr learned my addr 172.21.15.99:0/1260212701 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bce600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 0x7f0bc819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 -- 172.21.15.99:0/1260212701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 msgr2=0x7f0bc819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 0x7f0bc819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 -- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc8102210 msgr2=0x7f0bc81a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc8102210 0x7f0bc81a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bcd200700 1 -- 172.21.15.99:0/1260212701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc819b110 con 0x7f0bc8102bd0 2024-03-23T23:42:50.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.950+0000 7f0bce600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 0x7f0bc819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.951+0000 7f0bcd200700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000c00 tx=0x7f0bb400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.951+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4027070 con 0x7f0bc8102bd0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.951+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bb40056d0 con 0x7f0bc8102bd0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.951+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc819b3a0 con 0x7f0bc8102bd0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.951+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc81a76c0 con 0x7f0bc8102bd0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.952+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4022e10 con 0x7f0bc8102bd0 2024-03-23T23:42:50.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.952+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc8068e50 con 0x7f0bc8102bd0 2024-03-23T23:42:50.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.956+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0bb4009d10 con 0x7f0bc8102bd0 2024-03-23T23:42:50.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.957+0000 7f0bc6000700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 0x7f0bb00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:50.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.957+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0bb40cf760 con 0x7f0bc8102bd0 2024-03-23T23:42:50.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.957+0000 7f0bcdc00700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 0x7f0bb00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:50.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.958+0000 7f0bcdc00700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 0x7f0bb00a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bc819baf0 tx=0x7f0bbc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:50.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:50.961+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0bb4013070 con 0x7f0bc8102bd0 2024-03-23T23:42:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.306+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0bc81a78f0 con 0x7f0bc8102bd0 2024-03-23T23:42:51.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.306+0000 7f0bc6000700 1 -- 172.21.15.99:0/1260212701 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0bb409c810 con 0x7f0bc8102bd0 2024-03-23T23:42:51.307 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:51.307 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T23:42:51.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 msgr2=0x7f0bb00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 0x7f0bb00a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0bc819baf0 tx=0x7f0bbc00b040 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 msgr2=0x7f0bc81a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000c00 tx=0x7f0bb400bab0 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 shutdown_connections 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0bb00a2910 0x7f0bb00a4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc81082f0 0x7f0bc819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bc8102210 0x7f0bc81a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 --2- 172.21.15.99:0/1260212701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bc8102bd0 0x7f0bc81a0d30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.310+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 >> 172.21.15.99:0/1260212701 conn(0x7f0bc8079250 msgr2=0x7f0bc810d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.311+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 shutdown_connections 2024-03-23T23:42:51.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:51.311+0000 7f0bd0600700 1 -- 172.21.15.99:0/1260212701 wait complete. 2024-03-23T23:42:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:51 smithi099 ceph-mon[31715]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:42:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:51 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1260212701' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T23:42:52.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:51 smithi149 ceph-mon[36083]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:42:52.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:51 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1260212701' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T23:42:52.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:51 smithi174 ceph-mon[37002]: pgmap v369: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:42:52.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:51 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1260212701' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T23:42:52.245 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-23T23:42:52.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:42:52.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 -- 172.21.15.99:0/1127028217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 msgr2=0x7fe018073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 --2- 172.21.15.99:0/1127028217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe018073ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe008009900 tx=0x7fe00802bcd0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 -- 172.21.15.99:0/1127028217 shutdown_connections 2024-03-23T23:42:52.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 --2- 172.21.15.99:0/1127028217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe01810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 --2- 172.21.15.99:0/1127028217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe0180744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 --2- 172.21.15.99:0/1127028217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe018073ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 -- 172.21.15.99:0/1127028217 >> 172.21.15.99:0/1127028217 conn(0x7fe0180fcc80 msgr2=0x7fe0180ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:52.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 -- 172.21.15.99:0/1127028217 shutdown_connections 2024-03-23T23:42:52.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.457+0000 7fe01ea00640 1 -- 172.21.15.99:0/1127028217 wait complete. 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.458+0000 7fe01ea00640 1 Processor -- start 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.458+0000 7fe01ea00640 1 -- start start 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe018199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe0181a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01810f560 con 0x7fe018074080 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01810f3e0 con 0x7fe018109560 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ea00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01810f6e0 con 0x7fe018075660 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe018199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe0181a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39392/0 (socket says 172.21.15.99:39392) 2024-03-23T23:42:52.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe018199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43202/0 (socket says 172.21.15.99:43202) 2024-03-23T23:42:52.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017e00640 1 -- 172.21.15.99:0/3259811859 learned_addr learned my addr 172.21.15.99:0/3259811859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 -- 172.21.15.99:0/3259811859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 msgr2=0x7fe01819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 -- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 msgr2=0x7fe018199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe018199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe01ca00640 1 -- 172.21.15.99:0/3259811859 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0181a17d0 con 0x7fe018109560 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.459+0000 7fe017400640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe01ca00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe0181a1160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe010009fd0 tx=0x7fe01000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:52.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000ce20 con 0x7fe018109560 2024-03-23T23:42:52.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe010005020 con 0x7fe018109560 2024-03-23T23:42:52.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0181a1ac0 con 0x7fe018109560 2024-03-23T23:42:52.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010010720 con 0x7fe018109560 2024-03-23T23:42:52.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.460+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0181a1fc0 con 0x7fe018109560 2024-03-23T23:42:52.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.461+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe018108320 con 0x7fe018109560 2024-03-23T23:42:52.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.464+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe010005540 con 0x7fe018109560 2024-03-23T23:42:52.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.465+0000 7fe014c00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 0x7fdffc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.465+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0100cf1c0 con 0x7fe018109560 2024-03-23T23:42:52.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.465+0000 7fe017e00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 0x7fdffc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.466+0000 7fe017e00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 0x7fdffc0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe00802c7b0 tx=0x7fe00802c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:52.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.469+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe01009c1e0 con 0x7fe018109560 2024-03-23T23:42:52.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.698+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0180622a0 con 0x7fe018109560 2024-03-23T23:42:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.699+0000 7fe014c00640 1 -- 172.21.15.99:0/3259811859 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7fe01009bae0 con 0x7fe018109560 2024-03-23T23:42:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:52.700 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:42:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 msgr2=0x7fdffc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 0x7fdffc0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe00802c7b0 tx=0x7fe00802c920 comp rx=0 tx=0).stop 2024-03-23T23:42:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 msgr2=0x7fe0181a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe0181a1160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe010009fd0 tx=0x7fe01000ef10 comp rx=0 tx=0).stop 2024-03-23T23:42:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 shutdown_connections 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdffc0a2800 0x7fdffc0a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe018109560 0x7fe0181a1160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018075660 0x7fe01819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 --2- 172.21.15.99:0/3259811859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018074080 0x7fe018199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 >> 172.21.15.99:0/3259811859 conn(0x7fe0180fcc80 msgr2=0x7fe0181054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 shutdown_connections 2024-03-23T23:42:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.703+0000 7fe01ea00640 1 -- 172.21.15.99:0/3259811859 wait complete. 2024-03-23T23:42:52.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:42:52.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:42:52.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 -- 172.21.15.99:0/4180768304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 msgr2=0x7ff9dc106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 --2- 172.21.15.99:0/4180768304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc106da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009900 tx=0x7ff9c802bc30 comp rx=0 tx=0).stop 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 -- 172.21.15.99:0/4180768304 shutdown_connections 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 --2- 172.21.15.99:0/4180768304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 --2- 172.21.15.99:0/4180768304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc106da0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 --2- 172.21.15.99:0/4180768304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 -- 172.21.15.99:0/4180768304 >> 172.21.15.99:0/4180768304 conn(0x7ff9dc0fba10 msgr2=0x7ff9dc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 -- 172.21.15.99:0/4180768304 shutdown_connections 2024-03-23T23:42:52.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.970+0000 7ff9e1800640 1 -- 172.21.15.99:0/4180768304 wait complete. 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.971+0000 7ff9e1800640 1 Processor -- start 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.971+0000 7ff9e1800640 1 -- start start 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc10d250 con 0x7ff9dc102080 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc10d0d0 con 0x7ff9dc1072e0 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e1800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc10d3d0 con 0x7ff9dc1049b0 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9dbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9dbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43222/0 (socket says 172.21.15.99:43222) 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9dbe00640 1 -- 172.21.15.99:0/2272390461 learned_addr learned my addr 172.21.15.99:0/2272390461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.972+0000 7ff9e0e00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 -- 172.21.15.99:0/2272390461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 msgr2=0x7ff9dc197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 -- 172.21.15.99:0/2272390461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 msgr2=0x7ff9dc19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 -- 172.21.15.99:0/2272390461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc19f4f0 con 0x7ff9dc102080 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9db400640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:52.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9e0e00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9dbe00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e9f0 tx=0x7ff9cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00ccc0 con 0x7ff9dc102080 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc010910 con 0x7ff9dc102080 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc19f7e0 con 0x7ff9dc102080 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0188a0 con 0x7ff9dc102080 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.973+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc19fd00 con 0x7ff9dc102080 2024-03-23T23:42:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.976+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc0fd500 con 0x7ff9dc102080 2024-03-23T23:42:52.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.977+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff9cc010ac0 con 0x7ff9dc102080 2024-03-23T23:42:52.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.978+0000 7ff9d8c00640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 0x7ff9c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:52.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.978+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff9cc0cef40 con 0x7ff9dc102080 2024-03-23T23:42:52.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.978+0000 7ff9db400640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 0x7ff9c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.979+0000 7ff9db400640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 0x7ff9c00a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802c170 tx=0x7ff9c802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:52.983+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9cc09bf60 con 0x7ff9dc102080 2024-03-23T23:42:53.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:52 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3259811859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:53.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:52 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3259811859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:53.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:52 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3259811859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:53.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.181+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9dc02d070 con 0x7ff9dc102080 2024-03-23T23:42:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.182+0000 7ff9d8c00640 1 -- 172.21.15.99:0/2272390461 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9dc02d070 con 0x7ff9dc102080 2024-03-23T23:42:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 msgr2=0x7ff9c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 0x7ff9c00a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802c170 tx=0x7ff9c802df90 comp rx=0 tx=0).stop 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 msgr2=0x7ff9dc1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e9f0 tx=0x7ff9cc00edf0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 shutdown_connections 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9c00a2800 0x7ff9c00a4cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9dc1072e0 0x7ff9dc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9dc1049b0 0x7ff9dc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 --2- 172.21.15.99:0/2272390461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc102080 0x7ff9dc1978d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 >> 172.21.15.99:0/2272390461 conn(0x7ff9dc0fba10 msgr2=0x7ff9dc1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 shutdown_connections 2024-03-23T23:42:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.185+0000 7ff9e1800640 1 -- 172.21.15.99:0/2272390461 wait complete. 2024-03-23T23:42:53.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 -- 172.21.15.99:0/961106658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 msgr2=0x7f6870102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 --2- 172.21.15.99:0/961106658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870102c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f685c009860 tx=0x7f685c02bb70 comp rx=0 tx=0).stop 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 -- 172.21.15.99:0/961106658 shutdown_connections 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 --2- 172.21.15.99:0/961106658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 0x7f68701099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 --2- 172.21.15.99:0/961106658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870102c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 --2- 172.21.15.99:0/961106658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f68701019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.412+0000 7f6876000640 1 -- 172.21.15.99:0/961106658 >> 172.21.15.99:0/961106658 conn(0x7f68700fcb10 msgr2=0x7f68700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.413+0000 7f6876000640 1 -- 172.21.15.99:0/961106658 shutdown_connections 2024-03-23T23:42:53.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.413+0000 7f6876000640 1 -- 172.21.15.99:0/961106658 wait complete. 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 Processor -- start 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 -- start start 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f6870199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 0x7f68701a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687010d230 con 0x7f68701015a0 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687010d0b0 con 0x7f6870103150 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6876000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687010d3b0 con 0x7f6870102790 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6874c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f6870199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6867e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6867e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39430/0 (socket says 172.21.15.99:39430) 2024-03-23T23:42:53.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6874c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f6870199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43236/0 (socket says 172.21.15.99:43236) 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6867e00640 1 -- 172.21.15.99:0/914459500 learned_addr learned my addr 172.21.15.99:0/914459500 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.414+0000 7f6875600640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 0x7f68701a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 -- 172.21.15.99:0/914459500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 msgr2=0x7f68701a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 0x7f68701a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 -- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 msgr2=0x7f6870199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f6870199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 -- 172.21.15.99:0/914459500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68701a1690 con 0x7f6870102790 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6867e00640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f685c0349e0 tx=0x7f685c032450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c041030 con 0x7f6870102790 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c02c510 con 0x7f6870102790 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68701a1920 con 0x7f6870102790 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.415+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c039040 con 0x7f6870102790 2024-03-23T23:42:53.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.416+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68701a1e10 con 0x7f6870102790 2024-03-23T23:42:53.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.417+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68701a1fe0 con 0x7f6870102790 2024-03-23T23:42:53.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.419+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f685c044070 con 0x7f6870102790 2024-03-23T23:42:53.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.421+0000 7f686e000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 0x7f68540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.421+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f685c0efff0 con 0x7f6870102790 2024-03-23T23:42:53.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.421+0000 7f6874c00640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 0x7f68540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.422+0000 7f6874c00640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 0x7f68540a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f68701025f0 tx=0x7f6860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:53.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.422+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f685c0bd070 con 0x7f6870102790 2024-03-23T23:42:53.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.622+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68701a2370 con 0x7f6870102790 2024-03-23T23:42:53.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.622+0000 7f686e000640 1 -- 172.21.15.99:0/914459500 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f685c0bc970 con 0x7f6870102790 2024-03-23T23:42:53.623 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 msgr2=0x7f68540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 0x7f68540a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f68701025f0 tx=0x7f6860009260 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 msgr2=0x7f6870199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f685c0349e0 tx=0x7f685c032450 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 shutdown_connections 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68540a2800 0x7f68540a4cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6870103150 0x7f68701a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870102790 0x7f6870199fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 --2- 172.21.15.99:0/914459500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68701015a0 0x7f6870199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 >> 172.21.15.99:0/914459500 conn(0x7f68700fcb10 msgr2=0x7f68700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:53.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 shutdown_connections 2024-03-23T23:42:53.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.626+0000 7f6876000640 1 -- 172.21.15.99:0/914459500 wait complete. 2024-03-23T23:42:53.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:42:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 -- 172.21.15.99:0/4274889905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd378074100 msgr2=0x7fd378074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4274889905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd378074100 0x7fd378074580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd364009930 tx=0x7fd36402bc20 comp rx=0 tx=0).stop 2024-03-23T23:42:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 -- 172.21.15.99:0/4274889905 shutdown_connections 2024-03-23T23:42:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4274889905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37810db70 0x7fd37810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4274889905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd378074100 0x7fd378074580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4274889905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3780756e0 0x7fd378073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 -- 172.21.15.99:0/4274889905 >> 172.21.15.99:0/4274889905 conn(0x7fd3780fcb10 msgr2=0x7fd3780fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 -- 172.21.15.99:0/4274889905 shutdown_connections 2024-03-23T23:42:53.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.861+0000 7fd37ca00640 1 -- 172.21.15.99:0/4274889905 wait complete. 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.862+0000 7fd37ca00640 1 Processor -- start 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 -- start start 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3780756e0 0x7fd37819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37810db70 0x7fd3781a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd378113840 con 0x7fd37810db70 2024-03-23T23:42:53.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781136c0 con 0x7fd3780756e0 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd37ca00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781139c0 con 0x7fd378074100 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd376800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd376800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39448/0 (socket says 172.21.15.99:39448) 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd376800640 1 -- 172.21.15.99:0/4184492406 learned_addr learned my addr 172.21.15.99:0/4184492406 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd375e00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3780756e0 0x7fd37819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.863+0000 7fd377200640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37810db70 0x7fd3781a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 -- 172.21.15.99:0/4184492406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3780756e0 msgr2=0x7fd37819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3780756e0 0x7fd37819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 -- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37810db70 msgr2=0x7fd3781a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37810db70 0x7fd3781a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 -- 172.21.15.99:0/4184492406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3781a5ad0 con 0x7fd378074100 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd376800640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd36800ea60 tx=0x7fd36800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:53.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800cde0 con 0x7fd378074100 2024-03-23T23:42:53.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3680035f0 con 0x7fd378074100 2024-03-23T23:42:53.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3781a5dc0 con 0x7fd378074100 2024-03-23T23:42:53.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368010670 con 0x7fd378074100 2024-03-23T23:42:53.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.864+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3781a6220 con 0x7fd378074100 2024-03-23T23:42:53.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.866+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378103d40 con 0x7fd378074100 2024-03-23T23:42:53.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.867+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd3680037a0 con 0x7fd378074100 2024-03-23T23:42:53.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.868+0000 7fd36f400640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 0x7fd35c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:53.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.868+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd3680cee70 con 0x7fd378074100 2024-03-23T23:42:53.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.868+0000 7fd375e00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 0x7fd35c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:53.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.869+0000 7fd375e00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 0x7fd35c0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd364036040 tx=0x7fd364000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:53.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:53.877+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd36809be90 con 0x7fd378074100 2024-03-23T23:42:54.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.082+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3780622a0 con 0x7fd378074100 2024-03-23T23:42:54.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.083+0000 7fd36f400640 1 -- 172.21.15.99:0/4184492406 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd36809b790 con 0x7fd378074100 2024-03-23T23:42:54.083 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:42:54.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 msgr2=0x7fd35c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 0x7fd35c0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd364036040 tx=0x7fd364000bc0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 msgr2=0x7fd37819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd36800ea60 tx=0x7fd36800ef20 comp rx=0 tx=0).stop 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 shutdown_connections 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd35c0a2800 0x7fd35c0a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37810db70 0x7fd3781a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3780756e0 0x7fd37819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 --2- 172.21.15.99:0/4184492406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd378074100 0x7fd37819deb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 >> 172.21.15.99:0/4184492406 conn(0x7fd3780fcb10 msgr2=0x7fd378100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 shutdown_connections 2024-03-23T23:42:54.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.085+0000 7fd37ca00640 1 -- 172.21.15.99:0/4184492406 wait complete. 2024-03-23T23:42:54.101 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:53 smithi099 ceph-mon[31715]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:42:54.101 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2272390461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.101 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:53 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/914459500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.104 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:42:54.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:53 smithi149 ceph-mon[36083]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:42:54.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2272390461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:53 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/914459500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:53 smithi174 ceph-mon[37002]: pgmap v370: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:42:54.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2272390461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:53 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/914459500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:54.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 -- 172.21.15.99:0/2734293347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2881021c0 msgr2=0x7fd2881045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 --2- 172.21.15.99:0/2734293347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2881021c0 0x7fd2881045b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009900 tx=0x7fd27c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 -- 172.21.15.99:0/2734293347 shutdown_connections 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 --2- 172.21.15.99:0/2734293347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd288107420 0x7fd288109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 --2- 172.21.15.99:0/2734293347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288104af0 0x7fd288106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 --2- 172.21.15.99:0/2734293347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2881021c0 0x7fd2881045b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 -- 172.21.15.99:0/2734293347 >> 172.21.15.99:0/2734293347 conn(0x7fd2880fbbd0 msgr2=0x7fd2880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 -- 172.21.15.99:0/2734293347 shutdown_connections 2024-03-23T23:42:54.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.305+0000 7fd290000640 1 -- 172.21.15.99:0/2734293347 wait complete. 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.306+0000 7fd290000640 1 Processor -- start 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 -- start start 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 0x7fd288197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 0x7fd28819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28810d3c0 con 0x7fd288107420 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28810d240 con 0x7fd2881021c0 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd290000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28810d540 con 0x7fd288104af0 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd28d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd28d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51558/0 (socket says 172.21.15.99:51558) 2024-03-23T23:42:54.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd28d600640 1 -- 172.21.15.99:0/4036447993 learned_addr learned my addr 172.21.15.99:0/4036447993 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd28cc00640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 0x7fd288197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.307+0000 7fd28e000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 0x7fd28819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 -- 172.21.15.99:0/4036447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 msgr2=0x7fd288197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 0x7fd288197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 -- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 msgr2=0x7fd28819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 0x7fd28819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 -- 172.21.15.99:0/4036447993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28819f630 con 0x7fd2881021c0 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28cc00640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 0x7fd288197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28e000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 0x7fd28819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:54.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd28d600640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c043020 con 0x7fd2881021c0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28819f8c0 con 0x7fd2881021c0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd27c0054e0 con 0x7fd2881021c0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.308+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c003c40 con 0x7fd2881021c0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.309+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28819fe60 con 0x7fd2881021c0 2024-03-23T23:42:54.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.310+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880688e0 con 0x7fd2881021c0 2024-03-23T23:42:54.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.311+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd27c046020 con 0x7fd2881021c0 2024-03-23T23:42:54.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.314+0000 7fd286000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 0x7fd2700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.314+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd27c0f0bd0 con 0x7fd2881021c0 2024-03-23T23:42:54.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.314+0000 7fd28cc00640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 0x7fd2700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.315+0000 7fd28cc00640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 0x7fd2700a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2740095f0 tx=0x7fd274005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:54.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.317+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd27c04d020 con 0x7fd2881021c0 2024-03-23T23:42:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.618+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2880622a0 con 0x7fd2881021c0 2024-03-23T23:42:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.619+0000 7fd286000640 1 -- 172.21.15.99:0/4036447993 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fd27c0bdbf0 con 0x7fd2881021c0 2024-03-23T23:42:54.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:42:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:42:54.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 msgr2=0x7fd2700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 0x7fd2700a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2740095f0 tx=0x7fd274005f90 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 msgr2=0x7fd288197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c02c650 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 shutdown_connections 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd2700a2800 0x7fd2700a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd288107420 0x7fd28819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd288104af0 0x7fd288197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 --2- 172.21.15.99:0/4036447993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2881021c0 0x7fd288197a10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.622+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 >> 172.21.15.99:0/4036447993 conn(0x7fd2880fbbd0 msgr2=0x7fd288102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.623+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 shutdown_connections 2024-03-23T23:42:54.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.623+0000 7fd290000640 1 -- 172.21.15.99:0/4036447993 wait complete. 2024-03-23T23:42:54.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:42:54.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 -- 172.21.15.99:0/1519904186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 msgr2=0x7f4ed4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 --2- 172.21.15.99:0/1519904186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed4103b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009860 tx=0x7f4ec402bb70 comp rx=0 tx=0).stop 2024-03-23T23:42:54.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 -- 172.21.15.99:0/1519904186 shutdown_connections 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 --2- 172.21.15.99:0/1519904186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41052b0 0x7f4ed410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 --2- 172.21.15.99:0/1519904186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41048f0 0x7f4ed4104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 --2- 172.21.15.99:0/1519904186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed4103b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 -- 172.21.15.99:0/1519904186 >> 172.21.15.99:0/1519904186 conn(0x7f4ed40fec70 msgr2=0x7f4ed41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 -- 172.21.15.99:0/1519904186 shutdown_connections 2024-03-23T23:42:54.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.876+0000 7f4ed3200640 1 -- 172.21.15.99:0/1519904186 wait complete. 2024-03-23T23:42:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 Processor -- start 2024-03-23T23:42:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 -- start start 2024-03-23T23:42:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 0x7f4ed41a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f380 con 0x7f4ed4103700 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f200 con 0x7f4ed41052b0 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.877+0000 7f4ed3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f500 con 0x7f4ed41048f0 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 0x7f4ed41a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39486/0 (socket says 172.21.15.99:39486) 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 -- 172.21.15.99:0/3461864953 learned_addr learned my addr 172.21.15.99:0/3461864953 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:54.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1e00640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed41999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43284/0 (socket says 172.21.15.99:43284) 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 -- 172.21.15.99:0/3461864953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 msgr2=0x7f4ed41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 0x7f4ed41a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 -- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 msgr2=0x7f4ed41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 -- 172.21.15.99:0/3461864953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed41a15f0 con 0x7f4ed41048f0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed2800640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 0x7f4ed41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.878+0000 7f4ed1400640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e9c0 tx=0x7f4ec000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.879+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000cde0 con 0x7f4ed41048f0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.879+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ec00035f0 con 0x7f4ed41048f0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.879+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed41a18e0 con 0x7f4ed41048f0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.879+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0010690 con 0x7f4ed41048f0 2024-03-23T23:42:54.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.879+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a1e80 con 0x7f4ed41048f0 2024-03-23T23:42:54.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.880+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb40052c0 con 0x7f4ed41048f0 2024-03-23T23:42:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.882+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4ec0004020 con 0x7f4ed41048f0 2024-03-23T23:42:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.883+0000 7f4ecaa00640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 0x7f4eb80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.884+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ec00cebc0 con 0x7f4ed41048f0 2024-03-23T23:42:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.884+0000 7f4ed1e00640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 0x7f4eb80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:54.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.884+0000 7f4ed1e00640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 0x7f4eb80a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009830 tx=0x7f4ec4034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:54.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:54.888+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ec009bc40 con 0x7f4ed41048f0 2024-03-23T23:42:55.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.089+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4eb40055b0 con 0x7f4ed41048f0 2024-03-23T23:42:55.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.092+0000 7f4ecaa00640 1 -- 172.21.15.99:0/3461864953 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f4ec009b540 con 0x7f4ed41048f0 2024-03-23T23:42:55.092 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:55.092 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 msgr2=0x7f4eb80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 0x7f4eb80a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009830 tx=0x7f4ec4034ad0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 msgr2=0x7f4ed4199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e9c0 tx=0x7f4ec000ee80 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 shutdown_connections 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4eb80a2880 0x7f4eb80a4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed41052b0 0x7f4ed41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ed41048f0 0x7f4ed4199f10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.094+0000 7f4ed3200640 1 --2- 172.21.15.99:0/3461864953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ed4103700 0x7f4ed41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.095+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 >> 172.21.15.99:0/3461864953 conn(0x7f4ed40fec70 msgr2=0x7f4ed4107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.095+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 shutdown_connections 2024-03-23T23:42:55.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.095+0000 7f4ed3200640 1 -- 172.21.15.99:0/3461864953 wait complete. 2024-03-23T23:42:55.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:54 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4184492406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:55.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:54 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4036447993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:55.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:42:55.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:42:55.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:54 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4184492406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:55.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:54 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4036447993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:55.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:54 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4184492406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:55.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:54 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4036447993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:55.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 -- 172.21.15.99:0/1432483139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e81028a0 msgr2=0x7fc6e8102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 --2- 172.21.15.99:0/1432483139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e81028a0 0x7fc6e8102d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402bc40 comp rx=0 tx=0).stop 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 -- 172.21.15.99:0/1432483139 shutdown_connections 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 --2- 172.21.15.99:0/1432483139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e8103260 0x7fc6e8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 --2- 172.21.15.99:0/1432483139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e81028a0 0x7fc6e8102d20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 --2- 172.21.15.99:0/1432483139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81016b0 0x7fc6e8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 -- 172.21.15.99:0/1432483139 >> 172.21.15.99:0/1432483139 conn(0x7fc6e80fcc40 msgr2=0x7fc6e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 -- 172.21.15.99:0/1432483139 shutdown_connections 2024-03-23T23:42:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.313+0000 7fc6ed800640 1 -- 172.21.15.99:0/1432483139 wait complete. 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.314+0000 7fc6ed800640 1 Processor -- start 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 -- start start 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 0x7fc6e8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e810d380 con 0x7fc6e81028a0 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e810d200 con 0x7fc6e8103260 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6ed800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e810d500 con 0x7fc6e81016b0 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 0x7fc6e8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39502/0 (socket says 172.21.15.99:39502) 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e6000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43316/0 (socket says 172.21.15.99:43316) 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.315+0000 7fc6e6a00640 1 -- 172.21.15.99:0/487466655 learned_addr learned my addr 172.21.15.99:0/487466655 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 -- 172.21.15.99:0/487466655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 msgr2=0x7fc6e8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 -- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 msgr2=0x7fc6e8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 -- 172.21.15.99:0/487466655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8074860 con 0x7fc6e8103260 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e6a00640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e6000640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6e7400640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 0x7fc6e8073fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000ce20 con 0x7fc6e8103260 2024-03-23T23:42:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e0005020 con 0x7fc6e8103260 2024-03-23T23:42:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e81a1ae0 con 0x7fc6e8103260 2024-03-23T23:42:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.316+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0010720 con 0x7fc6e8103260 2024-03-23T23:42:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.317+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e81a2000 con 0x7fc6e8103260 2024-03-23T23:42:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.318+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e80688e0 con 0x7fc6e8103260 2024-03-23T23:42:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.321+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc6e0005540 con 0x7fc6e8103260 2024-03-23T23:42:55.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.322+0000 7fc6df400640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 0x7fc6cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.322+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc6e00cf210 con 0x7fc6e8103260 2024-03-23T23:42:55.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.323+0000 7fc6e6a00640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 0x7fc6cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.323+0000 7fc6e6a00640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 0x7fc6cc0a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80058c0 tx=0x7fc6d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:55.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.326+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6e009c230 con 0x7fc6e8103260 2024-03-23T23:42:55.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.528+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6e8062360 con 0x7fc6e8103260 2024-03-23T23:42:55.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.528+0000 7fc6df400640 1 -- 172.21.15.99:0/487466655 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6e009bb30 con 0x7fc6e8103260 2024-03-23T23:42:55.529 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 msgr2=0x7fc6cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 0x7fc6cc0a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80058c0 tx=0x7fc6d8005f90 comp rx=0 tx=0).stop 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 msgr2=0x7fc6e8073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 0x7fc6e8073fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009fd0 tx=0x7fc6e000ef10 comp rx=0 tx=0).stop 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 shutdown_connections 2024-03-23T23:42:55.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc6cc0a2800 0x7fc6cc0a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6e8103260 0x7fc6e8073fd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e81028a0 0x7fc6e8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 --2- 172.21.15.99:0/487466655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6e81016b0 0x7fc6e8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 >> 172.21.15.99:0/487466655 conn(0x7fc6e80fcc40 msgr2=0x7fc6e8105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 shutdown_connections 2024-03-23T23:42:55.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.531+0000 7fc6ed800640 1 -- 172.21.15.99:0/487466655 wait complete. 2024-03-23T23:42:55.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:42:55.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 -- 172.21.15.99:0/2263064936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b601072c0 msgr2=0x7f5b601096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 --2- 172.21.15.99:0/2263064936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b601072c0 0x7f5b601096b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009ca0 tx=0x7f5b5802c100 comp rx=0 tx=0).stop 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 -- 172.21.15.99:0/2263064936 shutdown_connections 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 --2- 172.21.15.99:0/2263064936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b601072c0 0x7f5b601096b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 --2- 172.21.15.99:0/2263064936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b60104990 0x7f5b60106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 --2- 172.21.15.99:0/2263064936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.791+0000 7f5b66400640 1 -- 172.21.15.99:0/2263064936 >> 172.21.15.99:0/2263064936 conn(0x7f5b600fba10 msgr2=0x7f5b600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.792+0000 7f5b66400640 1 -- 172.21.15.99:0/2263064936 shutdown_connections 2024-03-23T23:42:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.792+0000 7f5b66400640 1 -- 172.21.15.99:0/2263064936 wait complete. 2024-03-23T23:42:55.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.792+0000 7f5b66400640 1 Processor -- start 2024-03-23T23:42:55.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 -- start start 2024-03-23T23:42:55.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 0x7f5b60197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 0x7f5b6019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6010d210 con 0x7f5b60104990 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6010d090 con 0x7f5b60102060 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b66400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6010d390 con 0x7f5b601072c0 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 0x7f5b60197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b65000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b65a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 0x7f5b6019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b65000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51614/0 (socket says 172.21.15.99:51614) 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b65000640 1 -- 172.21.15.99:0/1676073234 learned_addr learned my addr 172.21.15.99:0/1676073234 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.793+0000 7f5b5fe00640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 0x7f5b60197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43346/0 (socket says 172.21.15.99:43346) 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 -- 172.21.15.99:0/1676073234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 msgr2=0x7f5b6019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 0x7f5b6019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 -- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 msgr2=0x7f5b60197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 0x7f5b60197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:55.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 -- 172.21.15.99:0/1676073234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6019f440 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65a00640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 0x7f5b6019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b65000640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400ea60 tx=0x7f5b5400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400cde0 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b540035f0 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54010670 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6019f730 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.794+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6019fcd0 con 0x7f5b60102060 2024-03-23T23:42:55.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.796+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b600fd500 con 0x7f5b60102060 2024-03-23T23:42:55.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.797+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5b540037a0 con 0x7f5b60102060 2024-03-23T23:42:55.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.798+0000 7f5b5d600640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 0x7f5b480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:55.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.799+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5b540ced40 con 0x7f5b60102060 2024-03-23T23:42:55.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.799+0000 7f5b5fe00640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 0x7f5b480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:55.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.799+0000 7f5b5fe00640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 0x7f5b480a4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b60198d30 tx=0x7f5b4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:55.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:55.803+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b5409bd60 con 0x7f5b60102060 2024-03-23T23:42:56.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.010+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b600622a0 con 0x7f5b60102060 2024-03-23T23:42:56.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.010+0000 7f5b5d600640 1 -- 172.21.15.99:0/1676073234 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b5409b660 con 0x7f5b60102060 2024-03-23T23:42:56.011 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:42:56.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 msgr2=0x7f5b480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 0x7f5b480a4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b60198d30 tx=0x7f5b4c005f90 comp rx=0 tx=0).stop 2024-03-23T23:42:56.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 msgr2=0x7f5b60197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400ea60 tx=0x7f5b5400ef20 comp rx=0 tx=0).stop 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 shutdown_connections 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5b480a2880 0x7f5b480a4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b601072c0 0x7f5b6019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b60104990 0x7f5b60197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 --2- 172.21.15.99:0/1676073234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b60102060 0x7f5b60197820 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 >> 172.21.15.99:0/1676073234 conn(0x7f5b600fba10 msgr2=0x7f5b601029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 shutdown_connections 2024-03-23T23:42:56.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.013+0000 7f5b66400640 1 -- 172.21.15.99:0/1676073234 wait complete. 2024-03-23T23:42:56.036 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:42:56.106 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:55 smithi099 ceph-mon[31715]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:42:56.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:55 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3461864953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:56.107 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:55 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/487466655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:55 smithi149 ceph-mon[36083]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:42:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:55 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3461864953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:56.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:55 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/487466655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:56.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:55 smithi174 ceph-mon[37002]: pgmap v371: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:42:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:55 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3461864953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:56.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:55 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/487466655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:42:56.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 -- 172.21.15.99:0/3889540494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c1015a0 msgr2=0x7fa49c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 --2- 172.21.15.99:0/3889540494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c1015a0 0x7fa49c1019c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:42:56.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 -- 172.21.15.99:0/3889540494 shutdown_connections 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 --2- 172.21.15.99:0/3889540494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c103150 0x7fa49c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 --2- 172.21.15.99:0/3889540494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c102790 0x7fa49c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 --2- 172.21.15.99:0/3889540494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c1015a0 0x7fa49c1019c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 -- 172.21.15.99:0/3889540494 >> 172.21.15.99:0/3889540494 conn(0x7fa49c0fcb10 msgr2=0x7fa49c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 -- 172.21.15.99:0/3889540494 shutdown_connections 2024-03-23T23:42:56.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.297+0000 7fa49b200640 1 -- 172.21.15.99:0/3889540494 wait complete. 2024-03-23T23:42:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.298+0000 7fa49b200640 1 Processor -- start 2024-03-23T23:42:56.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.298+0000 7fa49b200640 1 -- start start 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 0x7fa49c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 0x7fa49c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 0x7fa49c1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10d240 con 0x7fa49c1015a0 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10d0c0 con 0x7fa49c102790 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c10d3c0 con 0x7fa49c103150 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 0x7fa49c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 0x7fa49c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 0x7fa49c199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43354/0 (socket says 172.21.15.99:43354) 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499e00640 1 -- 172.21.15.99:0/3564983927 learned_addr learned my addr 172.21.15.99:0/3564983927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49a800640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 0x7fa49c1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 -- 172.21.15.99:0/3564983927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 msgr2=0x7fa49c1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 0x7fa49c1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 -- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 msgr2=0x7fa49c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 0x7fa49c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa499400640 1 -- 172.21.15.99:0/3564983927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c1a1690 con 0x7fa49c102790 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.299+0000 7fa49a800640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 0x7fa49c1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa499400640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 0x7fa49c199fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa48800e9f0 tx=0x7fa48800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800ccc0 con 0x7fa49c102790 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa488010910 con 0x7fa49c102790 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c1a1980 con 0x7fa49c102790 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488018960 con 0x7fa49c102790 2024-03-23T23:42:56.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.300+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1a1f20 con 0x7fa49c102790 2024-03-23T23:42:56.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.302+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c1a2150 con 0x7fa49c102790 2024-03-23T23:42:56.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.303+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa488010ac0 con 0x7fa49c102790 2024-03-23T23:42:56.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.305+0000 7fa492a00640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 0x7fa4800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.305+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa4880cee80 con 0x7fa49c102790 2024-03-23T23:42:56.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.306+0000 7fa499e00640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 0x7fa4800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.306+0000 7fa499e00640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 0x7fa4800a4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006010 tx=0x7fa48c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:56.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.308+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa48809bea0 con 0x7fa49c102790 2024-03-23T23:42:56.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.602+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa49c1a2320 con 0x7fa49c102790 2024-03-23T23:42:56.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.602+0000 7fa492a00640 1 -- 172.21.15.99:0/3564983927 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa48809b7a0 con 0x7fa49c102790 2024-03-23T23:42:56.603 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:42:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 msgr2=0x7fa4800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 0x7fa4800a4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006010 tx=0x7fa48c02df90 comp rx=0 tx=0).stop 2024-03-23T23:42:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 msgr2=0x7fa49c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 0x7fa49c199fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa48800e9f0 tx=0x7fa48800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 shutdown_connections 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4800a2880 0x7fa4800a4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c103150 0x7fa49c1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa49c102790 0x7fa49c199fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 --2- 172.21.15.99:0/3564983927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c1015a0 0x7fa49c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 >> 172.21.15.99:0/3564983927 conn(0x7fa49c0fcb10 msgr2=0x7fa49c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 shutdown_connections 2024-03-23T23:42:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.605+0000 7fa49b200640 1 -- 172.21.15.99:0/3564983927 wait complete. 2024-03-23T23:42:56.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:42:56.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.829+0000 7ff3ea600640 1 -- 172.21.15.99:0/2768967041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e40756a0 msgr2=0x7ff3e4073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.829+0000 7ff3ea600640 1 --2- 172.21.15.99:0/2768967041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e40756a0 0x7ff3e4073af0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009900 tx=0x7ff3d402bc50 comp rx=0 tx=0).stop 2024-03-23T23:42:56.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 -- 172.21.15.99:0/2768967041 shutdown_connections 2024-03-23T23:42:56.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 --2- 172.21.15.99:0/2768967041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e4109560 0x7ff3e410b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 --2- 172.21.15.99:0/2768967041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 --2- 172.21.15.99:0/2768967041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e40756a0 0x7ff3e4073af0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 -- 172.21.15.99:0/2768967041 >> 172.21.15.99:0/2768967041 conn(0x7ff3e40fcc80 msgr2=0x7ff3e40ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:56.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 -- 172.21.15.99:0/2768967041 shutdown_connections 2024-03-23T23:42:56.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.830+0000 7ff3ea600640 1 -- 172.21.15.99:0/2768967041 wait complete. 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.831+0000 7ff3ea600640 1 Processor -- start 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.831+0000 7ff3ea600640 1 -- start start 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 0x7ff3e41a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e410f540 con 0x7ff3e40740c0 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e410f3c0 con 0x7ff3e4109560 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3ea600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e410f6c0 con 0x7ff3e40756a0 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43368/0 (socket says 172.21.15.99:43368) 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e3400640 1 -- 172.21.15.99:0/3974077920 learned_addr learned my addr 172.21.15.99:0/3974077920 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39572/0 (socket says 172.21.15.99:39572) 2024-03-23T23:42:56.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e3e00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 0x7ff3e41a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 -- 172.21.15.99:0/3974077920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 msgr2=0x7ff3e41a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 0x7ff3e41a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 -- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 msgr2=0x7ff3e4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.832+0000 7ff3e2a00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3e2a00640 1 -- 172.21.15.99:0/3974077920 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e41a1770 con 0x7ff3e40756a0 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3e3e00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 0x7ff3e41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3e2a00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9f0 tx=0x7ff3d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000ccc0 con 0x7ff3e40756a0 2024-03-23T23:42:56.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3d0010910 con 0x7ff3e40756a0 2024-03-23T23:42:56.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0018960 con 0x7ff3e40756a0 2024-03-23T23:42:56.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e41a1a60 con 0x7ff3e40756a0 2024-03-23T23:42:56.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.833+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e41a2000 con 0x7ff3e40756a0 2024-03-23T23:42:56.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.835+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e41082e0 con 0x7ff3e40756a0 2024-03-23T23:42:56.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.837+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff3d0010ac0 con 0x7ff3e40756a0 2024-03-23T23:42:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.839+0000 7ff3dbe00640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 0x7ff3c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.839+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3d00ceec0 con 0x7ff3e40756a0 2024-03-23T23:42:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.839+0000 7ff3e3400640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 0x7ff3c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.840+0000 7ff3e3400640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 0x7ff3c80a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40096c0 tx=0x7ff3d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:56.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:56.842+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3d009bee0 con 0x7ff3e40756a0 2024-03-23T23:42:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:56 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1676073234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:56 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3564983927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:56 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1676073234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:56 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3564983927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:56 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1676073234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:56 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3564983927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:42:57.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.209+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3e402d450 con 0x7ff3e40756a0 2024-03-23T23:42:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.210+0000 7ff3dbe00640 1 -- 172.21.15.99:0/3974077920 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7ff3e402d450 con 0x7ff3e40756a0 2024-03-23T23:42:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:57.211 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:42:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:42:57.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.213+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 msgr2=0x7ff3c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.213+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 0x7ff3c80a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40096c0 tx=0x7ff3d4000bc0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.213+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 msgr2=0x7ff3e419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.213+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9f0 tx=0x7ff3d000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 shutdown_connections 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff3c80a2880 0x7ff3c80a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3e4109560 0x7ff3e41a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3e40756a0 0x7ff3e419a090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 --2- 172.21.15.99:0/3974077920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3e40740c0 0x7ff3e4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 >> 172.21.15.99:0/3974077920 conn(0x7ff3e40fcc80 msgr2=0x7ff3e41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 shutdown_connections 2024-03-23T23:42:57.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.214+0000 7ff3ea600640 1 -- 172.21.15.99:0/3974077920 wait complete. 2024-03-23T23:42:57.235 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.468+0000 7fedfce00640 1 -- 172.21.15.99:0/148310234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81003f0 msgr2=0x7fedf8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 --2- 172.21.15.99:0/148310234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81003f0 0x7fedf8100810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fede8009900 tx=0x7fede802bc50 comp rx=0 tx=0).stop 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 -- 172.21.15.99:0/148310234 shutdown_connections 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 --2- 172.21.15.99:0/148310234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf8101fa0 0x7fedf8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 --2- 172.21.15.99:0/148310234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf81015e0 0x7fedf8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 --2- 172.21.15.99:0/148310234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81003f0 0x7fedf8100810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 -- 172.21.15.99:0/148310234 >> 172.21.15.99:0/148310234 conn(0x7fedf80fb980 msgr2=0x7fedf80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:57.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 -- 172.21.15.99:0/148310234 shutdown_connections 2024-03-23T23:42:57.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.469+0000 7fedfce00640 1 -- 172.21.15.99:0/148310234 wait complete. 2024-03-23T23:42:57.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 Processor -- start 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 -- start start 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 0x7fedf8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8101fa0 0x7fedf81a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf810c0d0 con 0x7fedf8101fa0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf810bf50 con 0x7fedf81015e0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedfce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf810c250 con 0x7fedf81003f0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.470+0000 7fedf7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39580/0 (socket says 172.21.15.99:39580) 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf7400640 1 -- 172.21.15.99:0/3511522585 learned_addr learned my addr 172.21.15.99:0/3511522585 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 0x7fedf8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf7e00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8101fa0 0x7fedf81a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:57.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 -- 172.21.15.99:0/3511522585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 msgr2=0x7fedf8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 -- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8101fa0 msgr2=0x7fedf81a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8101fa0 0x7fedf81a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 -- 172.21.15.99:0/3511522585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf81a1640 con 0x7fedf81015e0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf7400640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedf6a00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 0x7fedf8199f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fede400e9f0 tx=0x7fede400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400ccc0 con 0x7fedf81015e0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fede4010910 con 0x7fedf81015e0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4018960 con 0x7fedf81015e0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.471+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf81a1930 con 0x7fedf81015e0 2024-03-23T23:42:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.472+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf81a1e30 con 0x7fedf81015e0 2024-03-23T23:42:57.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.473+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf81a2060 con 0x7fedf81015e0 2024-03-23T23:42:57.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.476+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fede4010ac0 con 0x7fedf81015e0 2024-03-23T23:42:57.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.478+0000 7fedefe00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 0x7feddc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:57.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.478+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fede40cedf0 con 0x7fedf81015e0 2024-03-23T23:42:57.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.478+0000 7fedf7400640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 0x7feddc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:57.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.479+0000 7fedf7400640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 0x7feddc0a4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fede802c900 tx=0x7fede8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:57.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.482+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fede409b7c0 con 0x7fedf81015e0 2024-03-23T23:42:57.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.746+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedf81a2800 con 0x7fedf81015e0 2024-03-23T23:42:57.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.748+0000 7fedefe00640 1 -- 172.21.15.99:0/3511522585 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7fede409f860 con 0x7fedf81015e0 2024-03-23T23:42:57.749 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:57.749 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 msgr2=0x7feddc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 0x7feddc0a4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fede802c900 tx=0x7fede8000bc0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 msgr2=0x7fedf8199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 0x7fedf8199f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fede400e9f0 tx=0x7fede400eeb0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 shutdown_connections 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7feddc0a2880 0x7feddc0a4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8101fa0 0x7fedf81a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf81015e0 0x7fedf8199f60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 --2- 172.21.15.99:0/3511522585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedf81003f0 0x7fedf8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:57.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 >> 172.21.15.99:0/3511522585 conn(0x7fedf80fb980 msgr2=0x7fedf80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:57.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 shutdown_connections 2024-03-23T23:42:57.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:57.752+0000 7fedfce00640 1 -- 172.21.15.99:0/3511522585 wait complete. 2024-03-23T23:42:57.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:42:58.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 -- 172.21.15.99:0/1766716881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108102700 msgr2=0x7ff108102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 --2- 172.21.15.99:0/1766716881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108102700 0x7ff108102b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009900 tx=0x7ff0f402bc30 comp rx=0 tx=0).stop 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 -- 172.21.15.99:0/1766716881 shutdown_connections 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 --2- 172.21.15.99:0/1766716881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 0x7ff108109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 --2- 172.21.15.99:0/1766716881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108102700 0x7ff108102b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 --2- 172.21.15.99:0/1766716881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108101510 0x7ff108101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 -- 172.21.15.99:0/1766716881 >> 172.21.15.99:0/1766716881 conn(0x7ff1080fca80 msgr2=0x7ff1080feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 -- 172.21.15.99:0/1766716881 shutdown_connections 2024-03-23T23:42:58.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.001+0000 7ff10de00640 1 -- 172.21.15.99:0/1766716881 wait complete. 2024-03-23T23:42:58.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 Processor -- start 2024-03-23T23:42:58.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 -- start start 2024-03-23T23:42:58.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108102700 0x7ff108197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 0x7ff10819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10810d1f0 con 0x7ff108102700 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10810d070 con 0x7ff108101510 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10de00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10810d370 con 0x7ff1081030c0 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff107e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108102700 0x7ff108197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.003+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51692/0 (socket says 172.21.15.99:51692) 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 -- 172.21.15.99:0/1110108751 learned_addr learned my addr 172.21.15.99:0/1110108751 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10d400640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 0x7ff10819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 -- 172.21.15.99:0/1110108751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 msgr2=0x7ff10819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 0x7ff10819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 -- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108102700 msgr2=0x7ff108197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108102700 0x7ff108197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 -- 172.21.15.99:0/1110108751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10819f410 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10ca00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00e9c0 tx=0x7ff0fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00cd80 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc0035f0 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc010690 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.004+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10819f700 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.005+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10819fbd0 con 0x7ff108101510 2024-03-23T23:42:58.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.006+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1081087d0 con 0x7ff108101510 2024-03-23T23:42:58.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.007+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff0fc0037a0 con 0x7ff108101510 2024-03-23T23:42:58.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.008+0000 7ff105600640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 0x7ff0f00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.009+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff0fc0cee10 con 0x7ff108101510 2024-03-23T23:42:58.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.009+0000 7ff107e00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 0x7ff0f00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.009+0000 7ff107e00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 0x7ff0f00a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402c240 tx=0x7ff0f402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.012+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0fc0d2030 con 0x7ff108101510 2024-03-23T23:42:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:57 smithi099 ceph-mon[31715]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:57 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3974077920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:57 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3511522585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:57 smithi149 ceph-mon[36083]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:57 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3974077920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:57 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3511522585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:57 smithi174 ceph-mon[37002]: pgmap v372: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:42:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:57 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3974077920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:57 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3511522585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:42:58.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.279+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff10802d070 con 0x7ff108101510 2024-03-23T23:42:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.280+0000 7ff105600640 1 -- 172.21.15.99:0/1110108751 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7ff0fc09be30 con 0x7ff108101510 2024-03-23T23:42:58.280 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:42:58.280 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:42:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 msgr2=0x7ff0f00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 0x7ff0f00a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff0f402c240 tx=0x7ff0f402df90 comp rx=0 tx=0).stop 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 msgr2=0x7ff1081977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00e9c0 tx=0x7ff0fc00ee80 comp rx=0 tx=0).stop 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 shutdown_connections 2024-03-23T23:42:58.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0f00a2880 0x7ff0f00a4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1081030c0 0x7ff10819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff108102700 0x7ff108197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 --2- 172.21.15.99:0/1110108751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff108101510 0x7ff1081977f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 >> 172.21.15.99:0/1110108751 conn(0x7ff1080fca80 msgr2=0x7ff1081059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 shutdown_connections 2024-03-23T23:42:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.284+0000 7ff10de00640 1 -- 172.21.15.99:0/1110108751 wait complete. 2024-03-23T23:42:58.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T23:41:09.728597+0000', 'modified': '2024-03-23T23:41:17.278370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24565, 'mds_2': 24557, 'mds_3': 24547, 'mds_4': 24526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/2919310963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2919310963}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2919310963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/4085463188', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 4085463188}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 4085463188}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1177424904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1177424904}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1177424904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/731827456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 731827456}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 731827456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/3655333447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 3655333447}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 3655333447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.174:6837/2363957261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2363957261}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2363957261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.174:6839/1404645838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 1404645838}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 1404645838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.99:6841/678226276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6840', 'nonce': 678226276}, {'type': 'v1', 'addr': '172.21.15.99:6841', 'nonce': 678226276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/4063961606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 4063961606}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 4063961606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/2464794793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2464794793}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2464794793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24547, 24526, 24557, 24569, 24565]} 2024-03-23T23:42:58.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T23:42:58.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T23:42:58.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 -- 172.21.15.99:0/2212132730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 msgr2=0x7f5360074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 --2- 172.21.15.99:0/2212132730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f5360074580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f534c009890 tx=0x7f534c02bb70 comp rx=0 tx=0).stop 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 -- 172.21.15.99:0/2212132730 shutdown_connections 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 --2- 172.21.15.99:0/2212132730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f536010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 --2- 172.21.15.99:0/2212132730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f5360074580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 --2- 172.21.15.99:0/2212132730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f5360073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 -- 172.21.15.99:0/2212132730 >> 172.21.15.99:0/2212132730 conn(0x7f53600fcb10 msgr2=0x7f53600fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 -- 172.21.15.99:0/2212132730 shutdown_connections 2024-03-23T23:42:58.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.521+0000 7f5366200640 1 -- 172.21.15.99:0/2212132730 wait complete. 2024-03-23T23:42:58.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.522+0000 7f5366200640 1 Processor -- start 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.522+0000 7f5366200640 1 -- start start 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f536019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f53601a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53601137d0 con 0x7f5360074100 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360113650 con 0x7f53600756e0 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5366200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360113950 con 0x7f536010db50 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5364e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f536019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5365800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f53601a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51702/0 (socket says 172.21.15.99:51702) 2024-03-23T23:42:58.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f5365800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f53601a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39630/0 (socket says 172.21.15.99:39630) 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 -- 172.21.15.99:0/3460356228 learned_addr learned my addr 172.21.15.99:0/3460356228 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 -- 172.21.15.99:0/3460356228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 msgr2=0x7f53601a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f53601a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 -- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 msgr2=0x7f536019def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f536019def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.523+0000 7f535fe00640 1 -- 172.21.15.99:0/3460356228 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53601a5b10 con 0x7f53600756e0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f535fe00640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f535d600640 1 -- 172.21.15.99:0/3460356228 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c041020 con 0x7f53600756e0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f535d600640 1 -- 172.21.15.99:0/3460356228 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f534c02c9f0 con 0x7f53600756e0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f535d600640 1 -- 172.21.15.99:0/3460356228 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c039040 con 0x7f53600756e0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53601a5da0 con 0x7f53600756e0 2024-03-23T23:42:58.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.524+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53601a61e0 con 0x7f53600756e0 2024-03-23T23:42:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.527+0000 7f535d600640 1 -- 172.21.15.99:0/3460356228 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f534c044020 con 0x7f53600756e0 2024-03-23T23:42:58.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.528+0000 7f535d600640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 0x7f53480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f535d600640 1 -- 172.21.15.99:0/3460356228 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f534c0f0460 con 0x7f53600756e0 2024-03-23T23:42:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f5364e00640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 0x7f53480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 Processor -- start 2024-03-23T23:42:58.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 -- start start 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a9490d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94912270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a94914d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94871c90 con 0x564a9490d770 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9481a980 con 0x564a949128e0 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.529+0000 7f536a58c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9486dd30 con 0x564a9490cd80 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5364e00640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 0x7f53480a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5354009bd0 tx=0x7f5354005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5364e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a9490d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a94914d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5364e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a9490d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39646/0 (socket says 172.21.15.99:39646) 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5364e00640 1 -- 172.21.15.99:0/309482567 learned_addr learned my addr 172.21.15.99:0/309482567 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f535fe00640 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94912270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 -- 172.21.15.99:0/309482567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 msgr2=0x564a9490d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a9490d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 -- 172.21.15.99:0/309482567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 msgr2=0x564a94912270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94912270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 -- 172.21.15.99:0/309482567 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a94832510 con 0x564a949128e0 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.530+0000 7f5365800640 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a94914d70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53580099b0 tx=0x7f53580095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0176e1fc423ab9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f5352000640 1 -- 172.21.15.99:0/309482567 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358003970 con 0x564a949128e0 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f5352000640 1 -- 172.21.15.99:0/309482567 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f535800d6b0 con 0x564a949128e0 2024-03-23T23:42:58.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f5352000640 1 -- 172.21.15.99:0/309482567 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358004350 con 0x564a949128e0 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 -- 172.21.15.99:0/309482567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 msgr2=0x564a94914d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a94914d70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53580099b0 tx=0x7f53580095c0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 -- 172.21.15.99:0/309482567 shutdown_connections 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a94914d70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94912270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 --2- 172.21.15.99:0/309482567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a9490d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 -- 172.21.15.99:0/309482567 >> 172.21.15.99:0/309482567 conn(0x564a949082f0 msgr2=0x564a949086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.531+0000 7f536a58c740 1 -- 172.21.15.99:0/309482567 shutdown_connections 2024-03-23T23:42:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 -- 172.21.15.99:0/309482567 wait complete. 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 -- start start 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 Processor -- start 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94917770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a949128e0 0x564a9491e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.532+0000 7f536a58c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94915890 con 0x564a9490d770 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f536a58c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9482ddd0 con 0x564a9490cd80 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f536a58c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94915a10 con 0x564a949128e0 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f536a58c740 10 client.0 _ll_register_callbacks cb 0x564a94585ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51720/0 (socket says 172.21.15.99:51720) 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 -- 172.21.15.99:0/2519656414 learned_addr learned my addr 172.21.15.99:0/2519656414 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5365800640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a949128e0 0x564a9491e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f535fe00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94917770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 -- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a949128e0 msgr2=0x564a9491e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a949128e0 0x564a9491e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 -- 172.21.15.99:0/2519656414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 msgr2=0x564a94917770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94917770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 -- 172.21.15.99:0/2519656414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a945862d0 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5364e00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5354003870 tx=0x7f5354004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725cc16985caa28c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.533+0000 7f5346a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400e720 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f535800d6b0 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f535400e8c0 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f535400eac0 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f536a58c740 10 client.24971 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a946506b0 con 0x564a9490cd80 2024-03-23T23:42:58.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.534+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f535800d6b0 con 0x564a9490cd80 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5354027420 con 0x564a9490cd80 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 10 client.24971 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 10 client.24971 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 20 client.24971 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a949239a0 0x564a94922a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 4 client.24971 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a94813520 con 0x564a949239a0 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.535+0000 7f535fe00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a949239a0 0x564a94922a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.536+0000 7f5346a00640 10 client.24971 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:42:58.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.536+0000 7f535fe00640 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a949239a0 0x564a94922a40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e06d1b9bb4ae45ca server_cookie=66db7f348d8c546d in_seq=0 out_seq=0 2024-03-23T23:42:58.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.536+0000 7f5346a00640 1 -- 172.21.15.99:0/2519656414 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a94813520 con 0x564a949239a0 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.536+0000 7f5346a00640 10 client.24971 handle_command_reply: tid=0 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.536+0000 7f536a58c740 1 client.24971 shutdown 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 msgr2=0x564a94917230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5354003870 tx=0x7f5354004730 comp rx=0 tx=0).stop 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 shutdown_connections 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a949128e0 0x564a9491e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a949239a0 0x564a94922a40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490d770 0x564a94917770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 --2- 172.21.15.99:0/2519656414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490cd80 0x564a94917230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f5346a00640 0 client.24971 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 >> 172.21.15.99:0/2519656414 conn(0x564a949082f0 msgr2=0x564a949074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 shutdown_connections 2024-03-23T23:42:58.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 1 -- 172.21.15.99:0/2519656414 wait complete. 2024-03-23T23:42:58.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.537+0000 7f536a58c740 20 client.24971 trim_cache size 0 max 16384 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 Processor -- start 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 -- start start 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 0x564a9491e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a949128e0 0x564a9490ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a949231c0 con 0x564a949128e0 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94923820 con 0x564a9490d770 2024-03-23T23:42:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f536a58c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94923e80 con 0x564a9490cd80 2024-03-23T23:42:58.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f5364e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f535fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 0x564a9491e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f5365800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a949128e0 0x564a9490ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f5364e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39664/0 (socket says 172.21.15.99:39664) 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.555+0000 7f5364e00640 1 -- 172.21.15.99:0/4268381657 learned_addr learned my addr 172.21.15.99:0/4268381657 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 -- 172.21.15.99:0/4268381657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 msgr2=0x564a9491e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 0x564a9491e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 -- 172.21.15.99:0/4268381657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a949128e0 msgr2=0x564a9490ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a949128e0 0x564a9490ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 -- 172.21.15.99:0/4268381657 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a948fa170 con 0x564a9490cd80 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f535fe00640 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 0x564a9491e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5364e00640 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5354027890 tx=0x7f535400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46bf7e7ef116dce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5351600640 1 -- 172.21.15.99:0/4268381657 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400ec60 con 0x564a9490cd80 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5351600640 1 -- 172.21.15.99:0/4268381657 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5354027a80 con 0x564a9490cd80 2024-03-23T23:42:58.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.556+0000 7f5351600640 1 -- 172.21.15.99:0/4268381657 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354031900 con 0x564a9490cd80 2024-03-23T23:42:58.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 -- 172.21.15.99:0/4268381657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 msgr2=0x564a949086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5354027890 tx=0x7f535400eef0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 -- 172.21.15.99:0/4268381657 shutdown_connections 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a949128e0 0x564a9490ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9490d770 0x564a9491e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 --2- 172.21.15.99:0/4268381657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490cd80 0x564a949086d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 -- 172.21.15.99:0/4268381657 >> 172.21.15.99:0/4268381657 conn(0x564a949082f0 msgr2=0x564a94922a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 -- 172.21.15.99:0/4268381657 shutdown_connections 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.557+0000 7f536a58c740 1 -- 172.21.15.99:0/4268381657 wait complete. 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.558+0000 7f536a58c740 1 -- start start 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.558+0000 7f536a58c740 1 Processor -- start 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490cd80 0x564a9491a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a9491dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94924b40 con 0x564a9490cd80 2024-03-23T23:42:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a949244e0 con 0x564a949128e0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a949251a0 con 0x564a9490d770 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f536a58c740 10 client.0 _ll_register_callbacks cb 0x7f5354004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f5364e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490cd80 0x564a9491a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39678/0 (socket says 172.21.15.99:39678) 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 -- 172.21.15.99:0/1980063330 learned_addr learned my addr 172.21.15.99:0/1980063330 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f5365800640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a9491dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 -- 172.21.15.99:0/1980063330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 msgr2=0x564a9491dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a9491dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 -- 172.21.15.99:0/1980063330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490cd80 msgr2=0x564a9491a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490cd80 0x564a9491a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.559+0000 7f535fe00640 1 -- 172.21.15.99:0/1980063330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a9491e300 con 0x564a9490d770 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f535fe00640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f534c0b91e0 tx=0x7f534c0bc810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d596f8b2feeae852 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0d3040 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f534c0bae20 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5354027a80 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f534c0c5520 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f536a58c740 10 client.25006 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a9497f260 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5354027a80 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f534c03c030 con 0x564a9490d770 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.560+0000 7f536a58c740 10 client.25006 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 10 client.25006 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 20 client.25006 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a94983a00 0x564a9497f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 4 client.25006 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a9487f1d0 con 0x564a94983a00 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f5364e00640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a94983a00 0x564a9497f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f5364e00640 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a94983a00 0x564a9497f5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9d85c536fcd3d0d server_cookie=879101b963cec2b9 in_seq=0 out_seq=0 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f5346a00640 10 client.25006 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f5346a00640 1 -- 172.21.15.99:0/1980063330 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564a9487f1d0 con 0x564a94983a00 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f5346a00640 10 client.25006 handle_command_reply: tid=0 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.561+0000 7f536a58c740 1 client.25006 shutdown 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 msgr2=0x564a9491acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f534c0b91e0 tx=0x7f534c0bc810 comp rx=0 tx=0).stop 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 shutdown_connections 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a949128e0 0x564a9491dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a9490d770 0x564a9491acf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x564a94983a00 0x564a9497f5a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 --2- 172.21.15.99:0/1980063330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a9490cd80 0x564a9491a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f5346a00640 0 client.25006 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 >> 172.21.15.99:0/1980063330 conn(0x564a949082f0 msgr2=0x564a949086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 shutdown_connections 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 1 -- 172.21.15.99:0/1980063330 wait complete. 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.562+0000 7f536a58c740 20 client.25006 trim_cache size 0 max 16384 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 4, 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24526, 2024-03-23T23:42:58.568 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 101.826307308, 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 131.90532262799999 2024-03-23T23:42:58.569 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 msgr2=0x7f53480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 0x7f53480a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5354009bd0 tx=0x7f5354005f90 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 msgr2=0x7f536019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c032430 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 shutdown_connections 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53480a2800 0x7f53480a4cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536010db50 0x7f53601a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53600756e0 0x7f536019e430 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 --2- 172.21.15.99:0/3460356228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5360074100 0x7f536019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 >> 172.21.15.99:0/3460356228 conn(0x7f53600fcb10 msgr2=0x7f5360100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 shutdown_connections 2024-03-23T23:42:58.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.563+0000 7f5366200640 1 -- 172.21.15.99:0/3460356228 wait complete. 2024-03-23T23:42:58.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T23:42:58.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 -- 172.21.15.99:0/3005878671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c100480 msgr2=0x7fb76c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 --2- 172.21.15.99:0/3005878671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c100480 0x7fb76c1008a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009ca0 tx=0x7fb75c02c100 comp rx=0 tx=0).stop 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 -- 172.21.15.99:0/3005878671 shutdown_connections 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 --2- 172.21.15.99:0/3005878671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c102030 0x7fb76c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 --2- 172.21.15.99:0/3005878671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c101670 0x7fb76c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 --2- 172.21.15.99:0/3005878671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c100480 0x7fb76c1008a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 -- 172.21.15.99:0/3005878671 >> 172.21.15.99:0/3005878671 conn(0x7fb76c0fba10 msgr2=0x7fb76c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 -- 172.21.15.99:0/3005878671 shutdown_connections 2024-03-23T23:42:58.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.786+0000 7fb76be00640 1 -- 172.21.15.99:0/3005878671 wait complete. 2024-03-23T23:42:58.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.788+0000 7fb76be00640 1 Processor -- start 2024-03-23T23:42:58.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.788+0000 7fb76be00640 1 -- start start 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c101670 0x7fb76c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c102030 0x7fb76c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c10c110 con 0x7fb76c101670 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c10bf90 con 0x7fb76c100480 2024-03-23T23:42:58.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c10c290 con 0x7fb76c102030 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c101670 0x7fb76c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51754/0 (socket says 172.21.15.99:51754) 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 -- 172.21.15.99:0/3036034536 learned_addr learned my addr 172.21.15.99:0/3036034536 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76b400640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c102030 0x7fb76c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 -- 172.21.15.99:0/3036034536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c102030 msgr2=0x7fb76c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c102030 0x7fb76c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 -- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c101670 msgr2=0x7fb76c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c101670 0x7fb76c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.789+0000 7fb76aa00640 1 -- 172.21.15.99:0/3036034536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c19f4c0 con 0x7fb76c100480 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb76aa00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000f80 tx=0x7fb75c035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb763400640 1 -- 172.21.15.99:0/3036034536 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c039040 con 0x7fb76c100480 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb763400640 1 -- 172.21.15.99:0/3036034536 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb75c044070 con 0x7fb76c100480 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c19f750 con 0x7fb76c100480 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb763400640 1 -- 172.21.15.99:0/3036034536 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0044a0 con 0x7fb76c100480 2024-03-23T23:42:58.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.790+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c19fc60 con 0x7fb76c100480 2024-03-23T23:42:58.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.793+0000 7fb763400640 1 -- 172.21.15.99:0/3036034536 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb75c004640 con 0x7fb76c100480 2024-03-23T23:42:58.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.793+0000 7fb774205740 1 Processor -- start 2024-03-23T23:42:58.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 -- start start 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9eeb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 0x562fb9eee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9e46c90 con 0x562fb9ee6b80 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9def980 con 0x562fb9eebcf0 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb774205740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9e42d30 con 0x562fb9ee6190 2024-03-23T23:42:58.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb76b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb76b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:39712/0 (socket says 172.21.15.99:39712) 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb76b400640 1 -- 172.21.15.99:0/2159403930 learned_addr learned my addr 172.21.15.99:0/2159403930 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.794+0000 7fb76aa00640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9eeb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76a000640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 0x562fb9eee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 -- 172.21.15.99:0/2159403930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 msgr2=0x562fb9eee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 0x562fb9eee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 -- 172.21.15.99:0/2159403930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 msgr2=0x562fb9eeb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9eeb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 -- 172.21.15.99:0/2159403930 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb9e07510 con 0x562fb9ee6190 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76a000640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 0x562fb9eee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb76b400640 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b6b0 tx=0x7fb76400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd318cfa4bb1027 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb763400640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 0x7fb7500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb757400640 1 -- 172.21.15.99:0/2159403930 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb764003bd0 con 0x562fb9ee6190 2024-03-23T23:42:58.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb757400640 1 -- 172.21.15.99:0/2159403930 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb764015070 con 0x562fb9ee6190 2024-03-23T23:42:58.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.795+0000 7fb757400640 1 -- 172.21.15.99:0/2159403930 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7640116a0 con 0x562fb9ee6190 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb76aa00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 0x7fb7500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb763400640 1 -- 172.21.15.99:0/3036034536 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb75c0f0fe0 con 0x7fb76c100480 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 -- 172.21.15.99:0/2159403930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 msgr2=0x562fb9ee65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b6b0 tx=0x7fb76400bb70 comp rx=0 tx=0).stop 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 -- 172.21.15.99:0/2159403930 shutdown_connections 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9eebcf0 0x562fb9eee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9eeb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 --2- 172.21.15.99:0/2159403930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9ee65b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 -- 172.21.15.99:0/2159403930 >> 172.21.15.99:0/2159403930 conn(0x562fb9edd2f0 msgr2=0x562fb9edd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb76aa00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 0x7fb7500a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0eee30 tx=0x7fb75c02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 -- 172.21.15.99:0/2159403930 shutdown_connections 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.796+0000 7fb774205740 1 -- 172.21.15.99:0/2159403930 wait complete. 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.797+0000 7fb774205740 1 -- start start 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.797+0000 7fb774205740 1 Processor -- start 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.797+0000 7fb774205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.797+0000 7fb774205740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb774205740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb774205740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9eeeca0 con 0x562fb9ee6b80 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb774205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9e02dd0 con 0x562fb9ee6190 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb774205740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9eeee20 con 0x562fb9eebcf0 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb774205740 10 client.0 _ll_register_callbacks cb 0x562fb9b5aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43476/0 (socket says 172.21.15.99:43476) 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51786/0 (socket says 172.21.15.99:51786) 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 -- 172.21.15.99:0/21592972 learned_addr learned my addr 172.21.15.99:0/21592972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76b400640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 -- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 msgr2=0x562fb9ef7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 -- 172.21.15.99:0/21592972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 msgr2=0x562fb9ef06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 -- 172.21.15.99:0/21592972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fb9b5b2d0 con 0x562fb9ee6b80 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76b400640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb76a000640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb758012700 tx=0x7fb758012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82be73734060d3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.798+0000 7fb74be00640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758004030 con 0x562fb9ee6b80 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb7580041d0 con 0x562fb9ee6b80 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb764015070 con 0x562fb9ee6b80 2024-03-23T23:42:58.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb758004410 con 0x562fb9ee6b80 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb774205740 10 client.14892 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562fb9c256b0 con 0x562fb9ee6b80 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.799+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb764015070 con 0x562fb9ee6b80 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb758003bf0 con 0x562fb9ee6b80 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 10 client.14892 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 10 client.14892 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 20 client.14892 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9efced0 0x562fb9efbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 4 client.14892 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fb9de8520 con 0x562fb9efced0 2024-03-23T23:42:58.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.800+0000 7fb76b400640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9efced0 0x562fb9efbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.801+0000 7fb74be00640 10 client.14892 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:42:58.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.801+0000 7fb76b400640 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9efced0 0x562fb9efbfd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=647eca83ee49c57c server_cookie=464e7ea97d82a836 in_seq=0 out_seq=0 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb74be00640 1 -- 172.21.15.99:0/21592972 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562fb9de8520 con 0x562fb9efced0 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb74be00640 10 client.14892 handle_command_reply: tid=0 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 client.14892 shutdown 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 msgr2=0x562fb9ef0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb758012700 tx=0x7fb758012bc0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 shutdown_connections 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9efced0 0x562fb9efbfd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6b80 0x562fb9ef0bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb74be00640 0 client.14892 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 --2- 172.21.15.99:0/21592972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6190 0x562fb9ef06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 >> 172.21.15.99:0/21592972 conn(0x562fb9edd2f0 msgr2=0x562fb9edc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 shutdown_connections 2024-03-23T23:42:58.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 1 -- 172.21.15.99:0/21592972 wait complete. 2024-03-23T23:42:58.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.802+0000 7fb774205740 20 client.14892 trim_cache size 0 max 16384 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 Processor -- start 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 -- start start 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9eeb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 0x562fb9ef0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efc6f0 con 0x562fb9eebcf0 2024-03-23T23:42:58.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efcd50 con 0x562fb9ee6b80 2024-03-23T23:42:58.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.820+0000 7fb774205740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efd3b0 con 0x562fb9ee6190 2024-03-23T23:42:58.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51796/0 (socket says 172.21.15.99:51796) 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9eeb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 -- 172.21.15.99:0/3886092880 learned_addr learned my addr 172.21.15.99:0/3886092880 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76b400640 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 0x562fb9ef0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 -- 172.21.15.99:0/3886092880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 msgr2=0x562fb9eeb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9eeb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 -- 172.21.15.99:0/3886092880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 msgr2=0x562fb9ef0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 0x562fb9ef0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 -- 172.21.15.99:0/3886092880 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb9ecf170 con 0x562fb9ee6b80 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76b400640 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 0x562fb9ef0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb76a000640 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7580126d0 tx=0x7fb7580096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92fb7e76ae6b9410 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb756a00640 1 -- 172.21.15.99:0/3886092880 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758004000 con 0x562fb9ee6b80 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.821+0000 7fb756a00640 1 -- 172.21.15.99:0/3886092880 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7580041a0 con 0x562fb9ee6b80 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb756a00640 1 -- 172.21.15.99:0/3886092880 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758003b50 con 0x562fb9ee6b80 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 -- 172.21.15.99:0/3886092880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 msgr2=0x562fb9ef7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7580126d0 tx=0x7fb7580096f0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 -- 172.21.15.99:0/3886092880 shutdown_connections 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9eebcf0 0x562fb9ef0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef7bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 --2- 172.21.15.99:0/3886092880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9ee6190 0x562fb9eeb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 -- 172.21.15.99:0/3886092880 >> 172.21.15.99:0/3886092880 conn(0x562fb9edd2f0 msgr2=0x562fb9efbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 -- 172.21.15.99:0/3886092880 shutdown_connections 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.822+0000 7fb774205740 1 -- 172.21.15.99:0/3886092880 wait complete. 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 -- start start 2024-03-23T23:42:58.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 Processor -- start 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 0x562fb9ef3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efe070 con 0x562fb9ee6190 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efda10 con 0x562fb9ee6b80 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb9efe6d0 con 0x562fb9eebcf0 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb774205740 10 client.0 _ll_register_callbacks cb 0x7fb758009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb76a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb76b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 0x562fb9ef3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb76a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:51802/0 (socket says 172.21.15.99:51802) 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb76b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 0x562fb9ef3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43486/0 (socket says 172.21.15.99:43486) 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.823+0000 7fb76a000640 1 -- 172.21.15.99:0/774309712 learned_addr learned my addr 172.21.15.99:0/774309712 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76b400640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 -- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 msgr2=0x562fb9ef71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 -- 172.21.15.99:0/774309712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 msgr2=0x562fb9ef3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 0x562fb9ef3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 -- 172.21.15.99:0/774309712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fb9ef77c0 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76b400640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb76a000640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb758009380 tx=0x7fb75800f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3995fb8a66c57b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb74be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758034070 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb75800fd40 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.824+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7580041a0 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb758040050 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 10 client.24995 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562fb9f588d0 con 0x562fb9ee6b80 2024-03-23T23:42:58.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7580041a0 con 0x562fb9ee6b80 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb758030480 con 0x562fb9ee6b80 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 10 client.24995 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 10 client.24995 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 20 client.24995 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9f5cb80 0x562fb9f58af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 4 client.24995 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562fb9e541d0 con 0x562fb9f5cb80 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.825+0000 7fb76b400640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9f5cb80 0x562fb9f58af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb76b400640 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9f5cb80 0x562fb9f58af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54d5095989612ce server_cookie=a75cdb84e52f3c4f in_seq=0 out_seq=0 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb74be00640 10 client.24995 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb74be00640 1 -- 172.21.15.99:0/774309712 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562fb9e541d0 con 0x562fb9f5cb80 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb74be00640 10 client.24995 handle_command_reply: tid=0 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 client.24995 shutdown 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 msgr2=0x562fb9ef41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb758009380 tx=0x7fb75800f9c0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 shutdown_connections 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562fb9f5cb80 0x562fb9f58af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fb9eebcf0 0x562fb9ef71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562fb9ee6b80 0x562fb9ef41b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb74be00640 0 client.24995 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 --2- 172.21.15.99:0/774309712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562fb9ee6190 0x562fb9ef3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 >> 172.21.15.99:0/774309712 conn(0x562fb9edd2f0 msgr2=0x562fb9eef1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 shutdown_connections 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 1 -- 172.21.15.99:0/774309712 wait complete. 2024-03-23T23:42:58.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.826+0000 7fb774205740 20 client.24995 trim_cache size 0 max 16384 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 3, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24547, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 103.10558328499999, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 126.373276316 2024-03-23T23:42:58.831 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:42:58.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 msgr2=0x7fb7500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 0x7fb7500a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0eee30 tx=0x7fb75c02c100 comp rx=0 tx=0).stop 2024-03-23T23:42:58.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 msgr2=0x7fb76c1978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000f80 tx=0x7fb75c035d10 comp rx=0 tx=0).stop 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 shutdown_connections 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7500a2880 0x7fb7500a4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb76c102030 0x7fb76c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c101670 0x7fb76c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 --2- 172.21.15.99:0/3036034536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb76c100480 0x7fb76c1978a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.827+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 >> 172.21.15.99:0/3036034536 conn(0x7fb76c0fba10 msgr2=0x7fb76c068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.828+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 shutdown_connections 2024-03-23T23:42:58.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:58.828+0000 7fb76be00640 1 -- 172.21.15.99:0/3036034536 wait complete. 2024-03-23T23:42:58.850 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T23:42:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 -- 172.21.15.99:0/3610630864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 msgr2=0x7f0f44101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 --2- 172.21.15.99:0/3610630864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44101ae0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009ca0 tx=0x7f0f3402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 -- 172.21.15.99:0/3610630864 shutdown_connections 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 --2- 172.21.15.99:0/3610630864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 0x7f0f44109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 --2- 172.21.15.99:0/3610630864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 0x7f0f44102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 --2- 172.21.15.99:0/3610630864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44101ae0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 -- 172.21.15.99:0/3610630864 >> 172.21.15.99:0/3610630864 conn(0x7f0f440fcc50 msgr2=0x7f0f440ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 -- 172.21.15.99:0/3610630864 shutdown_connections 2024-03-23T23:42:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.079+0000 7f0f49e00640 1 -- 172.21.15.99:0/3610630864 wait complete. 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.080+0000 7f0f49e00640 1 Processor -- start 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 -- start start 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 0x7f0f44073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 0x7f0f44073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4410d310 con 0x7f0f441028b0 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4410d190 con 0x7f0f44103270 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f49e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4410d490 con 0x7f0f441016c0 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f42a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f42a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35512/0 (socket says 172.21.15.99:35512) 2024-03-23T23:42:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f42a00640 1 -- 172.21.15.99:0/2165506743 learned_addr learned my addr 172.21.15.99:0/2165506743 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f42000640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 0x7f0f44073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.081+0000 7f0f43400640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 0x7f0f44073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 -- 172.21.15.99:0/2165506743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 msgr2=0x7f0f44073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 0x7f0f44073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 -- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 msgr2=0x7f0f44073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 0x7f0f44073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 -- 172.21.15.99:0/2165506743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44074860 con 0x7f0f441016c0 2024-03-23T23:42:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f42a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000f80 tx=0x7f0f34032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f48a00640 1 -- 172.21.15.99:0/2165506743 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34041020 con 0x7f0f441016c0 2024-03-23T23:42:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f48a00640 1 -- 172.21.15.99:0/2165506743 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f34034cd0 con 0x7f0f441016c0 2024-03-23T23:42:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f441a1ac0 con 0x7f0f441016c0 2024-03-23T23:42:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.082+0000 7f0f48a00640 1 -- 172.21.15.99:0/2165506743 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34039040 con 0x7f0f441016c0 2024-03-23T23:42:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.083+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441a1fe0 con 0x7f0f441016c0 2024-03-23T23:42:59.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.085+0000 7f0f48a00640 1 -- 172.21.15.99:0/2165506743 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0f34044020 con 0x7f0f441016c0 2024-03-23T23:42:59.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 Processor -- start 2024-03-23T23:42:59.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 -- start start 2024-03-23T23:42:59.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 0x5595056b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559505610c90 con 0x5595056b1700 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595055b9980 con 0x5595056ac590 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950560cd30 con 0x5595056abba0 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.086+0000 7f0f43400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35522/0 (socket says 172.21.15.99:35522) 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 -- 172.21.15.99:0/3634965966 learned_addr learned my addr 172.21.15.99:0/3634965966 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f42000640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 -- 172.21.15.99:0/3634965966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 msgr2=0x5595056b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f42a00640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 0x5595056b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 0x5595056b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 -- 172.21.15.99:0/3634965966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 msgr2=0x5595056b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 -- 172.21.15.99:0/3634965966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595055d1510 con 0x5595056abba0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f42000640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f42a00640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 0x5595056b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f43400640 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009c20 tx=0x7f0f3c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bd458ac34ef27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.087+0000 7f0f48a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 0x7f0f280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f48a00640 1 -- 172.21.15.99:0/2165506743 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0f340f08f0 con 0x7f0f441016c0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f2fe00640 1 -- 172.21.15.99:0/3634965966 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01e070 con 0x5595056abba0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f42a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 0x7f0f280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f2fe00640 1 -- 172.21.15.99:0/3634965966 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f3c019b10 con 0x5595056abba0 2024-03-23T23:42:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f2fe00640 1 -- 172.21.15.99:0/3634965966 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00eae0 con 0x5595056abba0 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3634965966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 msgr2=0x5595056abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f42a00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 0x7f0f280a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f340bcf10 tx=0x7f0f3402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009c20 tx=0x7f0f3c011810 comp rx=0 tx=0).stop 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3634965966 shutdown_connections 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056ac590 0x5595056b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3634965966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056abfc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.088+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3634965966 >> 172.21.15.99:0/3634965966 conn(0x5595056a7110 msgr2=0x5595056a7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.089+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3634965966 shutdown_connections 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.089+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3634965966 wait complete. 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.089+0000 7f0f4ca33740 1 -- start start 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.089+0000 7f0f4ca33740 1 Processor -- start 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.089+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056b1700 0x5595056bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056b46b0 con 0x5595056ac590 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595055ccdd0 con 0x5595056abba0 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056b4830 con 0x5595056b1700 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f4ca33740 10 client.0 _ll_register_callbacks cb 0x559505324ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f43400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f43400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39706/0 (socket says 172.21.15.99:39706) 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f43400640 1 -- 172.21.15.99:0/3378701239 learned_addr learned my addr 172.21.15.99:0/3378701239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f43400640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056b1700 0x5595056bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 -- 172.21.15.99:0/3378701239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056b1700 msgr2=0x5595056bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056b1700 0x5595056bd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 -- 172.21.15.99:0/3378701239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 msgr2=0x5595056b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.090+0000 7f0f42000640 1 -- 172.21.15.99:0/3378701239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595053252d0 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f42000640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056b6680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f440751c0 tx=0x7f0f30012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae00e56941a9cb06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30005220 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0f300053c0 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f3c019b10 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f300055c0 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f4ca33740 10 client.14910 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.091+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5595053ef6b0 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f3c019b10 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0f30010c70 con 0x5595056ac590 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 10 client.14910 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 10 client.14910 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 20 client.14910 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5595056c28d0 0x5595056c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 4 client.14910 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595055b2520 con 0x5595056c28d0 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.092+0000 7f0f43400640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5595056c28d0 0x5595056c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.093+0000 7f0f43400640 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5595056c28d0 0x5595056c1980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f21a182355e8ef3c server_cookie=816614f2a43c75df in_seq=0 out_seq=0 2024-03-23T23:42:59.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.093+0000 7f0f2cc00640 10 client.14910 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.094+0000 7f0f2cc00640 1 -- 172.21.15.99:0/3378701239 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5595055b2520 con 0x5595056c28d0 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.094+0000 7f0f2cc00640 10 client.14910 handle_command_reply: tid=0 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.094+0000 7f0f4ca33740 1 client.14910 shutdown 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 msgr2=0x5595056b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056b6680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f440751c0 tx=0x7f0f30012820 comp rx=0 tx=0).stop 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 shutdown_connections 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5595056c28d0 0x5595056c1980 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056b1700 0x5595056bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056b6680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/3378701239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f2cc00640 0 client.14910 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 >> 172.21.15.99:0/3378701239 conn(0x5595056a7110 msgr2=0x5595056a6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 shutdown_connections 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 1 -- 172.21.15.99:0/3378701239 wait complete. 2024-03-23T23:42:59.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.095+0000 7f0f4ca33740 20 client.14910 trim_cache size 0 max 16384 2024-03-23T23:42:59.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:58 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1110108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:59.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.123+0000 7f0f4ca33740 1 Processor -- start 2024-03-23T23:42:59.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 -- start start 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056b1700 0x5595056c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c20f0 con 0x5595056ac590 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c2750 con 0x5595056b1700 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c2db0 con 0x5595056abba0 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f42000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f42000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37432/0 (socket says 172.21.15.99:37432) 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f42000640 1 -- 172.21.15.99:0/4211801992 learned_addr learned my addr 172.21.15.99:0/4211801992 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f43400640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.124+0000 7f0f43400640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056b1700 0x5595056c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 -- 172.21.15.99:0/4211801992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 msgr2=0x5595056b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 -- 172.21.15.99:0/4211801992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056b1700 msgr2=0x5595056c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056b1700 0x5595056c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 -- 172.21.15.99:0/4211801992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595056b7270 con 0x5595056ac590 2024-03-23T23:42:59.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f43400640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f42000640 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f3000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb956c762fb4cb5c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f2f400640 1 -- 172.21.15.99:0/4211801992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30010910 con 0x5595056ac590 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f2f400640 1 -- 172.21.15.99:0/4211801992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f30026630 con 0x5595056ac590 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.125+0000 7f0f2f400640 1 -- 172.21.15.99:0/4211801992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3001c660 con 0x5595056ac590 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 -- 172.21.15.99:0/4211801992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 msgr2=0x5595056bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f3000b810 comp rx=0 tx=0).stop 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 -- 172.21.15.99:0/4211801992 shutdown_connections 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056b1700 0x5595056c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056ac590 0x5595056bd680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/4211801992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056abba0 0x5595056b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 -- 172.21.15.99:0/4211801992 >> 172.21.15.99:0/4211801992 conn(0x5595056a7110 msgr2=0x5595056ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 -- 172.21.15.99:0/4211801992 shutdown_connections 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.126+0000 7f0f4ca33740 1 -- 172.21.15.99:0/4211801992 wait complete. 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 -- start start 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 Processor -- start 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c3a70 con 0x5595056b1700 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c3410 con 0x5595056abba0 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595056c40d0 con 0x5595056ac590 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f4ca33740 10 client.0 _ll_register_callbacks cb 0x7f0f3000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f42000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f43400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f42000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35552/0 (socket says 172.21.15.99:35552) 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.127+0000 7f0f42000640 1 -- 172.21.15.99:0/1746768984 learned_addr learned my addr 172.21.15.99:0/1746768984 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f43400640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 -- 172.21.15.99:0/1746768984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 msgr2=0x5595056b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 -- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 msgr2=0x5595056bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 -- 172.21.15.99:0/1746768984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595056bd010 con 0x5595056ac590 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f43400640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f42000640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006010 tx=0x7f0f30026a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250ecc0ec8b31100 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3003e060 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f30026630 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.128+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0f30010b20 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f30010d60 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 10 client.25042 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55950571e090 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f30026630 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0f30031550 con 0x5595056ac590 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 10 client.25042 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 10 client.25042 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 20 client.25042 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x559505722360 0x55950571e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 4 client.25042 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.129+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55950561e1d0 con 0x559505722360 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f43400640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x559505722360 0x55950571e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f43400640 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x559505722360 0x55950571e260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25c8645cf74943f0 server_cookie=e33d4ae68d0f555b in_seq=0 out_seq=0 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f2cc00640 10 client.25042 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f2cc00640 1 -- 172.21.15.99:0/1746768984 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55950561e1d0 con 0x559505722360 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f2cc00640 10 client.25042 handle_command_reply: tid=0 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.130+0000 7f0f4ca33740 1 client.25042 shutdown 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 msgr2=0x5595056b9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006010 tx=0x7f0f30026a90 comp rx=0 tx=0).stop 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 shutdown_connections 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x559505722360 0x55950571e260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5595056b1700 0x5595056bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595056ac590 0x5595056b9a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 --2- 172.21.15.99:0/1746768984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595056abba0 0x5595056b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 >> 172.21.15.99:0/1746768984 conn(0x5595056a7110 msgr2=0x5595056c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f2cc00640 0 client.25042 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 shutdown_connections 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 1 -- 172.21.15.99:0/1746768984 wait complete. 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.131+0000 7f0f4ca33740 20 client.25042 trim_cache size 0 max 16384 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24557, 2024-03-23T23:42:59.134 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 104.419744157, 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 118.555910662 2024-03-23T23:42:59.135 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 msgr2=0x7f0f280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 0x7f0f280a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f340bcf10 tx=0x7f0f3402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 msgr2=0x7f0f44075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000f80 tx=0x7f0f34032ec0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 shutdown_connections 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0f280a2800 0x7f0f280a4cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f44103270 0x7f0f44073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f441028b0 0x7f0f44073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 --2- 172.21.15.99:0/2165506743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f441016c0 0x7f0f44075450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 >> 172.21.15.99:0/2165506743 conn(0x7f0f440fcc50 msgr2=0x7f0f44105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 shutdown_connections 2024-03-23T23:42:59.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.132+0000 7f0f49e00640 1 -- 172.21.15.99:0/2165506743 wait complete. 2024-03-23T23:42:59.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:58 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1110108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:59.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-23T23:42:59.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:58 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1110108751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 -- 172.21.15.99:0/1350330926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 msgr2=0x7f39e8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 --2- 172.21.15.99:0/1350330926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e8107fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009860 tx=0x7f39d802bb70 comp rx=0 tx=0).stop 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 -- 172.21.15.99:0/1350330926 shutdown_connections 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 --2- 172.21.15.99:0/1350330926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 0x7f39e8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 --2- 172.21.15.99:0/1350330926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 0x7f39e8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 --2- 172.21.15.99:0/1350330926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e8107fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.375+0000 7f39e7200640 1 -- 172.21.15.99:0/1350330926 >> 172.21.15.99:0/1350330926 conn(0x7f39e8076b70 msgr2=0x7f39e8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.376+0000 7f39e7200640 1 -- 172.21.15.99:0/1350330926 shutdown_connections 2024-03-23T23:42:59.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.376+0000 7f39e7200640 1 -- 172.21.15.99:0/1350330926 wait complete. 2024-03-23T23:42:59.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 Processor -- start 2024-03-23T23:42:59.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 -- start start 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 0x7f39e819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 0x7f39e81a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8113810 con 0x7f39e8108db0 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8113690 con 0x7f39e8107bc0 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8113990 con 0x7f39e8109770 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39732/0 (socket says 172.21.15.99:39732) 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e5e00640 1 -- 172.21.15.99:0/1805245374 learned_addr learned my addr 172.21.15.99:0/1805245374 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39dfe00640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 0x7f39e819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.377+0000 7f39e6800640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 0x7f39e81a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 -- 172.21.15.99:0/1805245374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 msgr2=0x7f39e81a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 0x7f39e81a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 -- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 msgr2=0x7f39e819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 0x7f39e819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 -- 172.21.15.99:0/1805245374 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e81a5ae0 con 0x7f39e8107bc0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e5e00640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39d80349e0 tx=0x7f39d8032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39dea00640 1 -- 172.21.15.99:0/1805245374 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8041020 con 0x7f39e8107bc0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39dea00640 1 -- 172.21.15.99:0/1805245374 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d802c9f0 con 0x7f39e8107bc0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e81a5d70 con 0x7f39e8107bc0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.378+0000 7f39dea00640 1 -- 172.21.15.99:0/1805245374 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8039040 con 0x7f39e8107bc0 2024-03-23T23:42:59.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.379+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e81a61f0 con 0x7f39e8107bc0 2024-03-23T23:42:59.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.382+0000 7f39efa41740 1 Processor -- start 2024-03-23T23:42:59.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.382+0000 7f39efa41740 1 -- start start 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0df3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0d52c90 con 0x555bb0df38e0 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0cfb980 con 0x555bb0dedd80 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39efa41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0d4ed30 con 0x555bb0dee770 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39744/0 (socket says 172.21.15.99:39744) 2024-03-23T23:42:59.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 -- 172.21.15.99:0/1201609827 learned_addr learned my addr 172.21.15.99:0/1201609827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39dfe00640 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39dea00640 1 -- 172.21.15.99:0/1805245374 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f39d8044020 con 0x7f39e8107bc0 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 -- 172.21.15.99:0/1201609827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 msgr2=0x555bb0df3270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0df3270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 -- 172.21.15.99:0/1201609827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 msgr2=0x555bb0df5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.383+0000 7f39e6800640 1 -- 172.21.15.99:0/1201609827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb0d13510 con 0x555bb0dedd80 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.384+0000 7f39e6800640 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e000e450 tx=0x7f39e00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afcb42a353ddaf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.384+0000 7f39d7400640 1 -- 172.21.15.99:0/1201609827 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000d910 con 0x555bb0dedd80 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.384+0000 7f39d7400640 1 -- 172.21.15.99:0/1201609827 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39e001c070 con 0x555bb0dedd80 2024-03-23T23:42:59.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.384+0000 7f39d7400640 1 -- 172.21.15.99:0/1201609827 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0016380 con 0x555bb0dedd80 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 -- 172.21.15.99:0/1201609827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 msgr2=0x555bb0dee1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e000e450 tx=0x7f39e00095c0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 -- 172.21.15.99:0/1201609827 shutdown_connections 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0df3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 --2- 172.21.15.99:0/1201609827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0dee1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 -- 172.21.15.99:0/1201609827 >> 172.21.15.99:0/1201609827 conn(0x555bb0de92f0 msgr2=0x555bb0de96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39dea00640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 0x7f39cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 -- 172.21.15.99:0/1201609827 shutdown_connections 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39efa41740 1 -- 172.21.15.99:0/1201609827 wait complete. 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39dea00640 1 -- 172.21.15.99:0/1805245374 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f39d80f0210 con 0x7f39e8107bc0 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.385+0000 7f39e6800640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 0x7f39cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 -- start start 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 Processor -- start 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39e6800640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 0x7f39cc0a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39e00179c0 tx=0x7f39e0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dee770 0x555bb0df8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0df38e0 0x555bb0dff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0df6890 con 0x555bb0dee770 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0d0edd0 con 0x555bb0df38e0 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.386+0000 7f39efa41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0df6a10 con 0x555bb0dedd80 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39efa41740 10 client.0 _ll_register_callbacks cb 0x555bb0a66ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0df38e0 0x555bb0dff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35582/0 (socket says 172.21.15.99:35582) 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 -- 172.21.15.99:0/2963104120 learned_addr learned my addr 172.21.15.99:0/2963104120 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dee770 0x555bb0df8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 -- 172.21.15.99:0/2963104120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0df38e0 msgr2=0x555bb0dff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0df38e0 0x555bb0dff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 -- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dee770 msgr2=0x555bb0df8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dee770 0x555bb0df8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 -- 172.21.15.99:0/2963104120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bb0a672d0 con 0x555bb0dedd80 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39e5e00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39d80bcb20 tx=0x7f39d80bc4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86df891a5fc84567 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:42:59.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.387+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80d5060 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f39d80b9200 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39e001c070 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d80c0ce0 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 10 client.25051 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555bb0b316b0 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39e001c070 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f39d803c030 con 0x555bb0dedd80 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 10 client.25051 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 10 client.25051 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 20 client.25051 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e049c0 0x555bb0e03930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 4 client.25051 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.388+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bb0cf4520 con 0x555bb0e049c0 2024-03-23T23:42:59.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.389+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e049c0 0x555bb0e03930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.389+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e049c0 0x555bb0e03930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8bd21d54eff24f7 server_cookie=137f904ae9b66c55 in_seq=0 out_seq=0 2024-03-23T23:42:59.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.389+0000 7f39cbe00640 10 client.25051 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39cbe00640 1 -- 172.21.15.99:0/2963104120 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555bb0cf4520 con 0x555bb0e049c0 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39cbe00640 10 client.25051 handle_command_reply: tid=0 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 client.25051 shutdown 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 msgr2=0x555bb0df8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39d80bcb20 tx=0x7f39d80bc4f0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 shutdown_connections 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e049c0 0x555bb0e03930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0df38e0 0x555bb0dff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dee770 0x555bb0df8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39cbe00640 0 client.25051 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:42:59.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 --2- 172.21.15.99:0/2963104120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dedd80 0x555bb0df8250 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.390+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 >> 172.21.15.99:0/2963104120 conn(0x555bb0de92f0 msgr2=0x555bb0de84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.391+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 shutdown_connections 2024-03-23T23:42:59.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.391+0000 7f39efa41740 1 -- 172.21.15.99:0/2963104120 wait complete. 2024-03-23T23:42:59.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.391+0000 7f39efa41740 20 client.25051 trim_cache size 0 max 16384 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.409+0000 7f39efa41740 1 Processor -- start 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.409+0000 7f39efa41740 1 -- start start 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.409+0000 7f39efa41740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.409+0000 7f39efa41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0dff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.409+0000 7f39efa41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39efa41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e041e0 con 0x555bb0df38e0 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39efa41740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e04840 con 0x555bb0dedd80 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39efa41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e04ea0 con 0x555bb0dee770 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39e6800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0dff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39750/0 (socket says 172.21.15.99:39750) 2024-03-23T23:42:59.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39e5e00640 1 -- 172.21.15.99:0/2052888222 learned_addr learned my addr 172.21.15.99:0/2052888222 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 -- 172.21.15.99:0/2052888222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 msgr2=0x555bb0de96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 -- 172.21.15.99:0/2052888222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 msgr2=0x555bb0df7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 -- 172.21.15.99:0/2052888222 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bb0ddb170 con 0x555bb0dee770 2024-03-23T23:42:59.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39e5e00640 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.410+0000 7f39dfe00640 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0dff760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x555bb0e06610 tx=0x7f39d0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a635a9e7854b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.411+0000 7f39d6a00640 1 -- 172.21.15.99:0/2052888222 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0023070 con 0x555bb0dee770 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.411+0000 7f39d6a00640 1 -- 172.21.15.99:0/2052888222 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d001f250 con 0x555bb0dee770 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.411+0000 7f39d6a00640 1 -- 172.21.15.99:0/2052888222 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d001f700 con 0x555bb0dee770 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- 172.21.15.99:0/2052888222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 msgr2=0x555bb0dff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0dff760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x555bb0e06610 tx=0x7f39d0005f90 comp rx=0 tx=0).stop 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- 172.21.15.99:0/2052888222 shutdown_connections 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0df38e0 0x555bb0df7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0dee770 0x555bb0dff760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 --2- 172.21.15.99:0/2052888222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dedd80 0x555bb0de96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- 172.21.15.99:0/2052888222 >> 172.21.15.99:0/2052888222 conn(0x555bb0de92f0 msgr2=0x555bb0def1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- 172.21.15.99:0/2052888222 shutdown_connections 2024-03-23T23:42:59.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- 172.21.15.99:0/2052888222 wait complete. 2024-03-23T23:42:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 -- start start 2024-03-23T23:42:59.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.412+0000 7f39efa41740 1 Processor -- start 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 0x555bb0dff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dee770 0x555bb0dff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e05b60 con 0x555bb0dedd80 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e05500 con 0x555bb0dee770 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bb0e061c0 con 0x555bb0df38e0 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39efa41740 10 client.0 _ll_register_callbacks cb 0x7f39d80bc960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39e6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39e5e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 0x555bb0dff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39dfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dee770 0x555bb0dff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39e6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35592/0 (socket says 172.21.15.99:35592) 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.413+0000 7f39e6800640 1 -- 172.21.15.99:0/3943261861 learned_addr learned my addr 172.21.15.99:0/3943261861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 -- 172.21.15.99:0/3943261861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dee770 msgr2=0x555bb0dff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dee770 0x555bb0dff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 -- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 msgr2=0x555bb0dff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 0x555bb0dff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 -- 172.21.15.99:0/3943261861 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bb0dfc4f0 con 0x555bb0df38e0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e5e00640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 0x555bb0dff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39e6800640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x555bb0deefd0 tx=0x7f39e00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958dd1bc4d547af0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0004c10 con 0x555bb0df38e0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f39e0004db0 con 0x555bb0df38e0 2024-03-23T23:42:59.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.414+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39d001f250 con 0x555bb0df38e0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e000d710 con 0x555bb0df38e0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 10 client.25057 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555bb0e60700 con 0x555bb0df38e0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d001f250 con 0x555bb0df38e0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f39e000b040 con 0x555bb0df38e0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 10 client.25057 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 10 client.25057 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 20 client.25057 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e64ae0 0x555bb0e609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 4 client.25057 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.415+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bb0d601d0 con 0x555bb0e64ae0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39e5e00640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e64ae0 0x555bb0e609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39e5e00640 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e64ae0 0x555bb0e609f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=879f5d0840ae52f8 server_cookie=80c830e8aae76c59 in_seq=0 out_seq=0 2024-03-23T23:42:59.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39cbe00640 10 client.25057 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:42:59.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39cbe00640 1 -- 172.21.15.99:0/3943261861 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x555bb0d601d0 con 0x555bb0e64ae0 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39cbe00640 10 client.25057 handle_command_reply: tid=0 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.416+0000 7f39efa41740 1 client.25057 shutdown 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 msgr2=0x555bb0dfbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x555bb0deefd0 tx=0x7f39e00095c0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 shutdown_connections 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x555bb0e64ae0 0x555bb0e609f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bb0df38e0 0x555bb0dfbcf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555bb0dee770 0x555bb0dff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39cbe00640 0 client.25057 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 --2- 172.21.15.99:0/3943261861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555bb0dedd80 0x555bb0dff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 >> 172.21.15.99:0/3943261861 conn(0x555bb0de92f0 msgr2=0x555bb0de96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 shutdown_connections 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 1 -- 172.21.15.99:0/3943261861 wait complete. 2024-03-23T23:42:59.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.417+0000 7f39efa41740 20 client.25057 trim_cache size 0 max 16384 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24565, 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 105.722262181, 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:42:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:42:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:42:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 122.83991612 2024-03-23T23:42:59.421 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:42:59.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 msgr2=0x7f39cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 0x7f39cc0a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39e00179c0 tx=0x7f39e0000f40 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 msgr2=0x7f39e819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39d80349e0 tx=0x7f39d8032430 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 shutdown_connections 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f39cc0a2880 0x7f39cc0a4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39e8109770 0x7f39e81a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e8108db0 0x7f39e819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 --2- 172.21.15.99:0/1805245374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39e8107bc0 0x7f39e819dec0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 >> 172.21.15.99:0/1805245374 conn(0x7f39e8076b70 msgr2=0x7f39e810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 shutdown_connections 2024-03-23T23:42:59.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.418+0000 7f39e7200640 1 -- 172.21.15.99:0/1805245374 wait complete. 2024-03-23T23:42:59.438 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.632+0000 7f53e8a00640 1 -- 172.21.15.99:0/4052526027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 msgr2=0x7f53e41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.632+0000 7f53e8a00640 1 --2- 172.21.15.99:0/4052526027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41019a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009900 tx=0x7f53d402bc30 comp rx=0 tx=0).stop 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 -- 172.21.15.99:0/4052526027 shutdown_connections 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 --2- 172.21.15.99:0/4052526027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4103130 0x7f53e41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 --2- 172.21.15.99:0/4052526027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4102770 0x7f53e4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 --2- 172.21.15.99:0/4052526027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41019a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 -- 172.21.15.99:0/4052526027 >> 172.21.15.99:0/4052526027 conn(0x7f53e40fcb10 msgr2=0x7f53e40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 -- 172.21.15.99:0/4052526027 shutdown_connections 2024-03-23T23:42:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.633+0000 7f53e8a00640 1 -- 172.21.15.99:0/4052526027 wait complete. 2024-03-23T23:42:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.634+0000 7f53e8a00640 1 Processor -- start 2024-03-23T23:42:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 -- start start 2024-03-23T23:42:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4102770 0x7f53e4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 0x7f53e419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e410d240 con 0x7f53e4102770 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e410d0c0 con 0x7f53e4101580 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e8a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e410d3c0 con 0x7f53e4103130 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4102770 0x7f53e4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39772/0 (socket says 172.21.15.99:39772) 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e2800640 1 -- 172.21.15.99:0/564947792 learned_addr learned my addr 172.21.15.99:0/564947792 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.635+0000 7f53e3200640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 0x7f53e419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 -- 172.21.15.99:0/564947792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 msgr2=0x7f53e419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 0x7f53e419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 -- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4102770 msgr2=0x7f53e4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4102770 0x7f53e4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 -- 172.21.15.99:0/564947792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e419f510 con 0x7f53e4101580 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e3200640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 0x7f53e419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.636+0000 7f53e2800640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000c00 tx=0x7f53d402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.637+0000 7f53db400640 1 -- 172.21.15.99:0/564947792 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4043020 con 0x7f53e4101580 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.637+0000 7f53db400640 1 -- 172.21.15.99:0/564947792 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d40054e0 con 0x7f53e4101580 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.637+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e419f7a0 con 0x7f53e4101580 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.637+0000 7f53db400640 1 -- 172.21.15.99:0/564947792 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4003ce0 con 0x7f53e4101580 2024-03-23T23:42:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.637+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e419fcc0 con 0x7f53e4101580 2024-03-23T23:42:59.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 Processor -- start 2024-03-23T23:42:59.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53db400640 1 -- 172.21.15.99:0/564947792 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f53d4046020 con 0x7f53e4101580 2024-03-23T23:42:59.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 -- start start 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4ab80 0x557033a4f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 0x557033a52180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570339aac90 con 0x557033a4a190 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033953980 con 0x557033a4fcf0 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53ec19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570339a6d30 con 0x557033a4ab80 2024-03-23T23:42:59.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53e3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53e2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4ab80 0x557033a4f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.640+0000 7f53e3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37518/0 (socket says 172.21.15.99:37518) 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 -- 172.21.15.99:0/3722427101 learned_addr learned my addr 172.21.15.99:0/3722427101 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 0x557033a52180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 -- 172.21.15.99:0/3722427101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4ab80 msgr2=0x557033a4f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4ab80 0x557033a4f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 -- 172.21.15.99:0/3722427101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 msgr2=0x557033a52180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 0x557033a52180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 -- 172.21.15.99:0/3722427101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55703396b510 con 0x557033a4a190 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 0x557033a52180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:59.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.641+0000 7f53e3200640 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0107e0 tx=0x7f53dc010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af193a1886081561 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53cf400640 1 -- 172.21.15.99:0/3722427101 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc003b70 con 0x557033a4a190 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53cf400640 1 -- 172.21.15.99:0/3722427101 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc00ee10 con 0x557033a4a190 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53cf400640 1 -- 172.21.15.99:0/3722427101 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0047a0 con 0x557033a4a190 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53db400640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 0x7f53c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53db400640 1 -- 172.21.15.99:0/564947792 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f53d40f0bb0 con 0x7f53e4101580 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53e2800640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 0x7f53c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 -- 172.21.15.99:0/3722427101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 msgr2=0x557033a4a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0107e0 tx=0x7f53dc010ca0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 -- 172.21.15.99:0/3722427101 shutdown_connections 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4fcf0 0x557033a52180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4ab80 0x557033a4f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.642+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3722427101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4a190 0x557033a4a5b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.643+0000 7f53ec19b740 1 -- 172.21.15.99:0/3722427101 >> 172.21.15.99:0/3722427101 conn(0x557033a45700 msgr2=0x557033a45b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.643+0000 7f53ec19b740 1 -- 172.21.15.99:0/3722427101 shutdown_connections 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.643+0000 7f53ec19b740 1 -- 172.21.15.99:0/3722427101 wait complete. 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.643+0000 7f53e2800640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 0x7f53c80a4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53d40b9930 tx=0x7f53d40dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.643+0000 7f53ec19b740 1 -- start start 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 Processor -- start 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4a190 0x557033a546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 0x557033a5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a52ca0 con 0x557033a4fcf0 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033966dd0 con 0x557033a4ab80 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a52e20 con 0x557033a4a190 2024-03-23T23:42:59.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53ec19b740 10 client.0 _ll_register_callbacks cb 0x5570336beac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53e3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4a190 0x557033a546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.644+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39796/0 (socket says 172.21.15.99:39796) 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 -- 172.21.15.99:0/3574591032 learned_addr learned my addr 172.21.15.99:0/3574591032 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e3200640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 0x557033a5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 -- 172.21.15.99:0/3574591032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4a190 msgr2=0x557033a546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4a190 0x557033a546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 -- 172.21.15.99:0/3574591032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 msgr2=0x557033a5bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 0x557033a5bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 -- 172.21.15.99:0/3574591032 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570336bf2d0 con 0x557033a4ab80 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e3200640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 0x557033a5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53d00126e0 tx=0x7f53d0012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d62f0d11fd530db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:42:59.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004050 con 0x557033a4ab80 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53dc00ee10 con 0x557033a4ab80 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.645+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f53d00041f0 con 0x557033a4ab80 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d00043f0 con 0x557033a4ab80 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 10 client.25040 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5570337896b0 con 0x557033a4ab80 2024-03-23T23:42:59.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53dc00ee10 con 0x557033a4ab80 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f53d0003c40 con 0x557033a4ab80 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 10 client.25040 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 10 client.25040 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 20 client.25040 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033a5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 4 client.25040 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.646+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55703394c520 con 0x557033a60ec0 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.647+0000 7f53e3200640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033a5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.647+0000 7f53e3200640 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033a5ffa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2833907081d1fe8e server_cookie=3af1b474393686f0 in_seq=0 out_seq=0 2024-03-23T23:42:59.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.647+0000 7f53c3e00640 10 client.25040 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53c3e00640 1 -- 172.21.15.99:0/3574591032 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55703394c520 con 0x557033a60ec0 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53c3e00640 10 client.25040 handle_command_reply: tid=0 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 client.25040 shutdown 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 msgr2=0x557033a54c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53d00126e0 tx=0x7f53d0012ba0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 shutdown_connections 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033a5ffa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53c3e00640 0 client.25040 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4fcf0 0x557033a5bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4ab80 0x557033a54c20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3574591032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4a190 0x557033a546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 >> 172.21.15.99:0/3574591032 conn(0x557033a45700 msgr2=0x557033a404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 shutdown_connections 2024-03-23T23:42:59.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.648+0000 7f53ec19b740 1 -- 172.21.15.99:0/3574591032 wait complete. 2024-03-23T23:42:59.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.649+0000 7f53ec19b740 20 client.25040 trim_cache size 0 max 16384 2024-03-23T23:42:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.666+0000 7f53ec19b740 1 Processor -- start 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.666+0000 7f53ec19b740 1 -- start start 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a606e0 con 0x557033a4ab80 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a60d40 con 0x557033a4a190 2024-03-23T23:42:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53ec19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a613a0 con 0x557033a4fcf0 2024-03-23T23:42:59.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37526/0 (socket says 172.21.15.99:37526) 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39810/0 (socket says 172.21.15.99:39810) 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 -- 172.21.15.99:0/1967533971 learned_addr learned my addr 172.21.15.99:0/1967533971 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e3200640 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 -- 172.21.15.99:0/1967533971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 msgr2=0x557033a491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 -- 172.21.15.99:0/1967533971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 msgr2=0x557033a4b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e1e00640 1 -- 172.21.15.99:0/1967533971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557033a4b410 con 0x557033a4ab80 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.667+0000 7f53e3200640 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.668+0000 7f53e1e00640 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53d00104d0 tx=0x7f53d0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ba2f98b62e6fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.668+0000 7f53cea00640 1 -- 172.21.15.99:0/1967533971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004000 con 0x557033a4ab80 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.668+0000 7f53cea00640 1 -- 172.21.15.99:0/1967533971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d000fe00 con 0x557033a4ab80 2024-03-23T23:42:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.668+0000 7f53cea00640 1 -- 172.21.15.99:0/1967533971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00108a0 con 0x557033a4ab80 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 -- 172.21.15.99:0/1967533971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 msgr2=0x557033a5bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53d00104d0 tx=0x7f53d0009f00 comp rx=0 tx=0).stop 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 -- 172.21.15.99:0/1967533971 shutdown_connections 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a5bc20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 --2- 172.21.15.99:0/1967533971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a4b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 -- 172.21.15.99:0/1967533971 >> 172.21.15.99:0/1967533971 conn(0x557033a45700 msgr2=0x557033a5ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 -- 172.21.15.99:0/1967533971 shutdown_connections 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.669+0000 7f53ec19b740 1 -- 172.21.15.99:0/1967533971 wait complete. 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.670+0000 7f53ec19b740 1 -- start start 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.670+0000 7f53ec19b740 1 Processor -- start 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a57990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a5aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a62060 con 0x557033a4ab80 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a61a00 con 0x557033a4a190 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557033a626c0 con 0x557033a4fcf0 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53ec19b740 10 client.0 _ll_register_callbacks cb 0x7f53d0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a57990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a57990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39812/0 (socket says 172.21.15.99:39812) 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e3200640 1 -- 172.21.15.99:0/3239809060 learned_addr learned my addr 172.21.15.99:0/3239809060 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e3200640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a5aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 -- 172.21.15.99:0/3239809060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 msgr2=0x557033a5aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a5aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 -- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 msgr2=0x557033a57990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a57990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e1e00640 1 -- 172.21.15.99:0/3239809060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557033a5b4e0 con 0x557033a4ab80 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.671+0000 7f53e3200640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a5aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53e1e00640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a57ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53d00058c0 tx=0x7f53d000fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b46b794f264aa7a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:42:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d003d060 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f53d0010850 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d000fe00 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d0005690 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53ec19b740 10 client.14952 fetch_fsmap learned FSMap version 26 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.672+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557033abc4b0 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53d000fe00 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f53d0038460 con 0x557033a4ab80 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 10 client.14952 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 10 client.14952 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 20 client.14952 populate_metadata read hostname 'smithi099' 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033abc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 4 client.14952 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:42:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5570339b81d0 con 0x557033a60ec0 2024-03-23T23:42:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.673+0000 7f53e3200640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033abc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53e3200640 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033abc6e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950f29f858deafc6 server_cookie=56a26f0f29ca6872 in_seq=0 out_seq=0 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53c3e00640 10 client.14952 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53c3e00640 1 -- 172.21.15.99:0/3239809060 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5570339b81d0 con 0x557033a60ec0 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53c3e00640 10 client.14952 handle_command_reply: tid=0 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 client.14952 shutdown 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 msgr2=0x557033a57ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a57ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53d00058c0 tx=0x7f53d000fb50 comp rx=0 tx=0).stop 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 shutdown_connections 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x557033a60ec0 0x557033abc6e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.674+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557033a4fcf0 0x557033a5aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557033a4ab80 0x557033a57ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 1 --2- 172.21.15.99:0/3239809060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557033a4a190 0x557033a57990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 >> 172.21.15.99:0/3239809060 conn(0x557033a45700 msgr2=0x557033a4b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53c3e00640 0 client.14952 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 shutdown_connections 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 1 -- 172.21.15.99:0/3239809060 wait complete. 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53ec19b740 20 client.14952 trim_cache size 0 max 16384 2024-03-23T23:42:59.678 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24569, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 109.017243272, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 116.319370924 2024-03-23T23:42:59.679 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 msgr2=0x7f53c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 0x7f53c80a4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53d40b9930 tx=0x7f53d40dd040 comp rx=0 tx=0).stop 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 msgr2=0x7f53e41978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.675+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000c00 tx=0x7f53d402c6c0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 shutdown_connections 2024-03-23T23:42:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53c80a2800 0x7f53c80a4cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53e4103130 0x7f53e419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e4102770 0x7f53e4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 --2- 172.21.15.99:0/564947792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e4101580 0x7f53e41978f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 >> 172.21.15.99:0/564947792 conn(0x7f53e40fcb10 msgr2=0x7f53e4105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 shutdown_connections 2024-03-23T23:42:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:42:59.676+0000 7f53e8a00640 1 -- 172.21.15.99:0/564947792 wait complete. 2024-03-23T23:42:59.691 INFO:tasks.cephadm:Setup complete, yielding 2024-03-23T23:42:59.691 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T23:42:59.701 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:42:59.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-23T23:43:00.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:42:59 smithi099 ceph-mon[31715]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:00.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:42:59 smithi149 ceph-mon[36083]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:00.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:42:59 smithi174 ceph-mon[37002]: pgmap v373: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:00.182 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:02.143 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:01 smithi149 ceph-mon[36083]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:02.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:01 smithi174 ceph-mon[37002]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:02.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:01 smithi099 ceph-mon[31715]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 -- 172.21.15.99:0/1293500270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8105d80 msgr2=0x7f4de8106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 --2- 172.21.15.99:0/1293500270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8105d80 0x7f4de8106200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009f90 tx=0x7f4de000b330 comp rx=0 tx=0).stop 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 -- 172.21.15.99:0/1293500270 shutdown_connections 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 --2- 172.21.15.99:0/1293500270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8105d80 0x7f4de8106200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 --2- 172.21.15.99:0/1293500270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8100a40 0x7f4de8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 --2- 172.21.15.99:0/1293500270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 -- 172.21.15.99:0/1293500270 >> 172.21.15.99:0/1293500270 conn(0x7f4de80fc660 msgr2=0x7f4de80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 -- 172.21.15.99:0/1293500270 shutdown_connections 2024-03-23T23:43:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.566+0000 7f4ded800700 1 -- 172.21.15.99:0/1293500270 wait complete. 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.567+0000 7f4ded800700 1 Processor -- start 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 -- start start 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 0x7f4de81a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8114350 con 0x7f4de81033b0 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81141d0 con 0x7f4de8105d80 2024-03-23T23:43:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4ded800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de81144d0 con 0x7f4de8100a40 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4de6a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:35658/0 (socket says 172.21.15.99:35658) 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4de6a00700 1 -- 172.21.15.99:0/2111996956 learned_addr learned my addr 172.21.15.99:0/2111996956 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4de7400700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 0x7f4de81a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.568+0000 7f4de6000700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 -- 172.21.15.99:0/2111996956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 msgr2=0x7f4de81a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 0x7f4de81a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 -- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 msgr2=0x7f4de819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 -- 172.21.15.99:0/2111996956 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de81a65d0 con 0x7f4de8100a40 2024-03-23T23:43:02.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6000700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:02.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de7400700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 0x7f4de81a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:02.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4de6a00700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800e9f0 tx=0x7f4dd800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:02.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800cbd0 con 0x7f4de8100a40 2024-03-23T23:43:02.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd800cd70 con 0x7f4de8100a40 2024-03-23T23:43:02.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8010430 con 0x7f4de8100a40 2024-03-23T23:43:02.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.569+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de81a68c0 con 0x7f4de8100a40 2024-03-23T23:43:02.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.570+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81a6e50 con 0x7f4de8100a40 2024-03-23T23:43:02.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.571+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8068e50 con 0x7f4de8100a40 2024-03-23T23:43:02.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.572+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4dd8004750 con 0x7f4de8100a40 2024-03-23T23:43:02.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.574+0000 7f4ddf400700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 0x7f4dcc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:02.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.575+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4dd80cebf0 con 0x7f4de8100a40 2024-03-23T23:43:02.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.575+0000 7f4de6000700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 0x7f4dcc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:02.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.575+0000 7f4de6000700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 0x7f4dcc0a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400bc10 tx=0x7f4dd400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:02.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.578+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4dd809bc10 con 0x7f4de8100a40 2024-03-23T23:43:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.875+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4de80625d0 con 0x7f4dcc0a2910 2024-03-23T23:43:02.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.876+0000 7f4ddf400700 1 -- 172.21.15.99:0/2111996956 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4de80625d0 con 0x7f4dcc0a2910 2024-03-23T23:43:02.877 INFO:teuthology.orchestra.run.smithi099.stdout:Backend: cephadm 2024-03-23T23:43:02.877 INFO:teuthology.orchestra.run.smithi099.stdout:Available: Yes 2024-03-23T23:43:02.877 INFO:teuthology.orchestra.run.smithi099.stdout:Paused: No 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 msgr2=0x7f4dcc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 0x7f4dcc0a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400bc10 tx=0x7f4dd400b5f0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 msgr2=0x7f4de819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800e9f0 tx=0x7f4dd800ed00 comp rx=0 tx=0).stop 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 shutdown_connections 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4dcc0a2910 0x7f4dcc0a4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4de8105d80 0x7f4de81a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4de8100a40 0x7f4de819e970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 --2- 172.21.15.99:0/2111996956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4de81033b0 0x7f4de819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:02.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.880+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 >> 172.21.15.99:0/2111996956 conn(0x7f4de80fc660 msgr2=0x7f4de8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:02.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.881+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 shutdown_connections 2024-03-23T23:43:02.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:02.881+0000 7f4ded800700 1 -- 172.21.15.99:0/2111996956 wait complete. 2024-03-23T23:43:03.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-23T23:43:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:03 smithi099 ceph-mon[31715]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:04.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:03 smithi099 ceph-mon[31715]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:03 smithi149 ceph-mon[36083]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:04.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:03 smithi149 ceph-mon[36083]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:03 smithi174 ceph-mon[37002]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:04.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:03 smithi174 ceph-mon[37002]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:04.279 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:06.031 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:05 smithi099 ceph-mon[31715]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:05 smithi149 ceph-mon[36083]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:06.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:05 smithi174 ceph-mon[37002]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:06.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 -- 172.21.15.99:0/1751796344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 msgr2=0x7fc2d4104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 --2- 172.21.15.99:0/1751796344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d4104810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009920 tx=0x7fc2c4009c30 comp rx=0 tx=0).stop 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 -- 172.21.15.99:0/1751796344 shutdown_connections 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 --2- 172.21.15.99:0/1751796344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d4105f90 0x7fc2d410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 --2- 172.21.15.99:0/1751796344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d4104810 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 --2- 172.21.15.99:0/1751796344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d41055f0 0x7fc2d4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.594+0000 7fc2db000700 1 -- 172.21.15.99:0/1751796344 >> 172.21.15.99:0/1751796344 conn(0x7fc2d40ff9b0 msgr2=0x7fc2d4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.595+0000 7fc2db000700 1 -- 172.21.15.99:0/1751796344 shutdown_connections 2024-03-23T23:43:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.595+0000 7fc2db000700 1 -- 172.21.15.99:0/1751796344 wait complete. 2024-03-23T23:43:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.596+0000 7fc2db000700 1 Processor -- start 2024-03-23T23:43:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.596+0000 7fc2db000700 1 -- start start 2024-03-23T23:43:06.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.596+0000 7fc2db000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.596+0000 7fc2db000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 0x7fc2d419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2db000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 0x7fc2d41a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2db000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d4110030 con 0x7fc2d4105f90 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2db000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d410feb0 con 0x7fc2d41043f0 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2db000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d41101b0 con 0x7fc2d41055f0 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:39866/0 (socket says 172.21.15.99:39866) 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 -- 172.21.15.99:0/3423349704 learned_addr learned my addr 172.21.15.99:0/3423349704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3400700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 0x7fc2d419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d9000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 0x7fc2d41a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 -- 172.21.15.99:0/3423349704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 msgr2=0x7fc2d419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 0x7fc2d419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 -- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 msgr2=0x7fc2d41a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 0x7fc2d41a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3e00700 1 -- 172.21.15.99:0/3423349704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d41a2300 con 0x7fc2d41043f0 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d3400700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 0x7fc2d419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.597+0000 7fc2d9000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 0x7fc2d41a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2d3e00700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005250 tx=0x7fc2c4005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4027070 con 0x7fc2d41043f0 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c400f930 con 0x7fc2d41043f0 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d41a2590 con 0x7fc2d41043f0 2024-03-23T23:43:06.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004b90 con 0x7fc2d41043f0 2024-03-23T23:43:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.598+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d41a29e0 con 0x7fc2d41043f0 2024-03-23T23:43:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.599+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d4068e50 con 0x7fc2d41043f0 2024-03-23T23:43:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.601+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc2c402a070 con 0x7fc2d41043f0 2024-03-23T23:43:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.602+0000 7fc2d0c00700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 0x7fc2b80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:06.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.602+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc2c40cf530 con 0x7fc2d41043f0 2024-03-23T23:43:06.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.603+0000 7fc2d3400700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 0x7fc2b80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:06.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.603+0000 7fc2d3400700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 0x7fc2b80a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009680 tx=0x7fc2c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.606+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2c405e740 con 0x7fc2d41043f0 2024-03-23T23:43:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.960+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2d40625d0 con 0x7fc2b80a2910 2024-03-23T23:43:06.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.977+0000 7fc2d0c00700 1 -- 172.21.15.99:0/3423349704 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7fc2d40625d0 con 0x7fc2b80a2910 2024-03-23T23:43:06.978 INFO:teuthology.orchestra.run.smithi099.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T23:43:06.978 INFO:teuthology.orchestra.run.smithi099.stdout:mds.a smithi099 running (2m) 26s ago 2m 16.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 beb5a84fadda 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.b smithi149 running (2m) 55s ago 2m 25.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 41a8fa1b1554 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.c smithi174 running (2m) 55s ago 2m 20.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 71029bed2fe8 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.d smithi099 running (2m) 26s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 972cf0b8722e 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.e smithi149 running (2m) 55s ago 2m 18.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ae5b11d1a3ad 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.f smithi174 running (2m) 55s ago 2m 16.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6c97a7bd053e 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.g smithi099 running (2m) 26s ago 2m 17.0M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6964d9a17ad3 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.h smithi149 running (2m) 55s ago 2m 20.1M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 d6fb423a1508 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.i smithi174 running (2m) 55s ago 2m 20.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 1a08a22abf3e 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.j smithi099 running (2m) 26s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6365bce062e6 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.k smithi149 running (2m) 55s ago 2m 16.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e4f25c19cb5e 2024-03-23T23:43:06.979 INFO:teuthology.orchestra.run.smithi099.stdout:mds.l smithi174 running (2m) 55s ago 2m 16.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b85a40263aea 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.x smithi099 *:9283,8765 running (11m) 26s ago 11m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 71f4dee37fba 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.y smithi149 *:8443,8765 running (8m) 55s ago 8m 452M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3b22133feda8 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.z smithi174 *:8443,8765 running (8m) 55s ago 8m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 a1ba38b568eb 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mon.a smithi099 running (11m) 26s ago 11m 48.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 57a017c5170b 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mon.b smithi149 running (8m) 55s ago 8m 41.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 1f9bb8b46399 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:mon.c smithi174 running (9m) 55s ago 9m 43.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 863c6d10876a 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 smithi099 running (8m) 26s ago 8m 54.8M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e0e1a505ad9f 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 smithi149 running (7m) 55s ago 7m 58.8M 1451M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5b121b74a781 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 smithi174 running (7m) 55s ago 7m 59.5M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e510c03b241e 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 smithi099 running (6m) 26s ago 6m 52.4M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e179d3108dd4 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 smithi149 running (6m) 55s ago 6m 52.9M 1451M 19.0.0-2145-g3be9a5c9 c664afe56bb0 ff66324ef80f 2024-03-23T23:43:06.980 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 smithi174 running (5m) 55s ago 5m 54.8M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 1df859644610 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 smithi099 running (5m) 26s ago 5m 41.3M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 25853cc901c1 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 smithi149 running (4m) 55s ago 4m 57.3M 1451M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e176ed49bad8 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 smithi174 running (4m) 55s ago 4m 44.4M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3afa35e544ce 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 smithi099 running (3m) 26s ago 3m 42.7M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 62e3876d12e5 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 smithi149 running (3m) 55s ago 3m 56.6M 1451M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f3b2b2d26dd4 2024-03-23T23:43:06.981 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 smithi174 running (3m) 55s ago 3m 52.5M 4270M 19.0.0-2145-g3be9a5c9 c664afe56bb0 fd34b8cb2c65 2024-03-23T23:43:06.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 msgr2=0x7fc2b80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:06.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 0x7fc2b80a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009680 tx=0x7fc2c0009260 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 msgr2=0x7fc2d419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005250 tx=0x7fc2c4005280 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 shutdown_connections 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc2b80a2910 0x7fc2b80a4dd0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009680 tx=0x7fc2c0009260 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4105f90 0x7fc2d41a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2d41043f0 0x7fc2d419a6a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 --2- 172.21.15.99:0/3423349704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2d41055f0 0x7fc2d419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 >> 172.21.15.99:0/3423349704 conn(0x7fc2d40ff9b0 msgr2=0x7fc2d4108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 shutdown_connections 2024-03-23T23:43:06.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:06.982+0000 7fc2db000700 1 -- 172.21.15.99:0/3423349704 wait complete. 2024-03-23T23:43:07.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-23T23:43:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:07 smithi149 ceph-mon[36083]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:07 smithi149 ceph-mon[36083]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:08.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:07 smithi174 ceph-mon[37002]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:08.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:07 smithi174 ceph-mon[37002]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:08.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:07 smithi099 ceph-mon[31715]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:08.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:07 smithi099 ceph-mon[31715]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:08.432 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:10.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:09 smithi149 ceph-mon[36083]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:10.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:09 smithi174 ceph-mon[37002]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:10.244 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:09 smithi099 ceph-mon[31715]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 -- 172.21.15.99:0/3134871574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4103460 msgr2=0x7fd5e41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 --2- 172.21.15.99:0/3134871574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4103460 0x7fd5e41038c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009990 tx=0x7fd5d0009ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 -- 172.21.15.99:0/3134871574 shutdown_connections 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 --2- 172.21.15.99:0/3134871574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e4103e00 0x7fd5e410a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 --2- 172.21.15.99:0/3134871574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4100480 0x7fd5e41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 --2- 172.21.15.99:0/3134871574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4103460 0x7fd5e41038c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 -- 172.21.15.99:0/3134871574 >> 172.21.15.99:0/3134871574 conn(0x7fd5e40fbe00 msgr2=0x7fd5e40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:10.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 -- 172.21.15.99:0/3134871574 shutdown_connections 2024-03-23T23:43:10.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.785+0000 7fd5ec600700 1 -- 172.21.15.99:0/3134871574 wait complete. 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.787+0000 7fd5ec600700 1 Processor -- start 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.787+0000 7fd5ec600700 1 -- start start 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4103460 0x7fd5e4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e4103e00 0x7fd5e4101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e410ddc0 con 0x7fd5e4103e00 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e410dc40 con 0x7fd5e4103460 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5ec600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e410df40 con 0x7fd5e4100480 2024-03-23T23:43:10.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:10.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:46700/0 (socket says 172.21.15.99:46700) 2024-03-23T23:43:10.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 -- 172.21.15.99:0/812266618 learned_addr learned my addr 172.21.15.99:0/812266618 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:10.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9200700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4103460 0x7fd5e4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 -- 172.21.15.99:0/812266618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4103460 msgr2=0x7fd5e4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4103460 0x7fd5e4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 -- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e4103e00 msgr2=0x7fd5e4101950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e4103e00 0x7fd5e4101950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.788+0000 7fd5e9c00700 1 -- 172.21.15.99:0/812266618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4102200 con 0x7fd5e4100480 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5e9c00700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e970 tx=0x7fd5d800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800cbd0 con 0x7fd5e4100480 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5d8004500 con 0x7fd5e4100480 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e41a2560 con 0x7fd5e4100480 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003dd0 con 0x7fd5e4100480 2024-03-23T23:43:10.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.789+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e41a2aa0 con 0x7fd5e4100480 2024-03-23T23:43:10.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.790+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e4068e50 con 0x7fd5e4100480 2024-03-23T23:43:10.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.794+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd5d8004020 con 0x7fd5e4100480 2024-03-23T23:43:10.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.796+0000 7fd5e2000700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 0x7fd5cc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:10.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.797+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd5d80ced50 con 0x7fd5e4100480 2024-03-23T23:43:10.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.797+0000 7fd5e9200700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 0x7fd5cc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:10.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.797+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5d80cf1d0 con 0x7fd5e4100480 2024-03-23T23:43:10.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:10.797+0000 7fd5e9200700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 0x7fd5cc0a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b2e0 tx=0x7fd5d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:11.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.051+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e40625d0 con 0x7fd5cc0a2910 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.066+0000 7fd5e2000700 1 -- 172.21.15.99:0/812266618 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fd5e40625d0 con 0x7fd5cc0a2910 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:mds.all 12/12 59s ago 2m smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:mgr 3/3 59s ago 8m smithi099=x;smithi149=y;smithi174=z;count:3 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:mon 3/3 59s ago 9m smithi099:172.21.15.99=a;smithi149:172.21.15.149=b;smithi174:172.21.15.174=c;count:3 2024-03-23T23:43:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:osd 12 59s ago - 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 msgr2=0x7fd5cc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 0x7fd5cc0a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b2e0 tx=0x7fd5d0005fd0 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 msgr2=0x7fd5e4102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e970 tx=0x7fd5d800ec80 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 shutdown_connections 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd5cc0a2910 0x7fd5cc0a4dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5e4103e00 0x7fd5e4101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5e4100480 0x7fd5e4102dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 --2- 172.21.15.99:0/812266618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5e4103460 0x7fd5e4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.068+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 >> 172.21.15.99:0/812266618 conn(0x7fd5e40fbe00 msgr2=0x7fd5e41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.069+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 shutdown_connections 2024-03-23T23:43:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:11.069+0000 7fd5ec600700 1 -- 172.21.15.99:0/812266618 wait complete. 2024-03-23T23:43:11.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-23T23:43:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:11 smithi149 ceph-mon[36083]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:11 smithi149 ceph-mon[36083]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:11 smithi174 ceph-mon[37002]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:12.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:11 smithi174 ceph-mon[37002]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:12.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:11 smithi099 ceph-mon[31715]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:12.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:11 smithi099 ceph-mon[31715]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:12.389 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:14.099 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:13 smithi099 ceph-mon[31715]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:13 smithi149 ceph-mon[36083]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:14.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:13 smithi174 ceph-mon[37002]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:14.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 -- 172.21.15.99:0/3179146621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 msgr2=0x7f82cc075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:14.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 --2- 172.21.15.99:0/3179146621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc075070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009990 tx=0x7f82b8009ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 -- 172.21.15.99:0/3179146621 shutdown_connections 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 --2- 172.21.15.99:0/3179146621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 0x7f82cc10c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 --2- 172.21.15.99:0/3179146621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 --2- 172.21.15.99:0/3179146621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc075070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 -- 172.21.15.99:0/3179146621 >> 172.21.15.99:0/3179146621 conn(0x7f82cc0fd6a0 msgr2=0x7f82cc0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 -- 172.21.15.99:0/3179146621 shutdown_connections 2024-03-23T23:43:14.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.636+0000 7f82d2400700 1 -- 172.21.15.99:0/3179146621 wait complete. 2024-03-23T23:43:14.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.638+0000 7f82d2400700 1 Processor -- start 2024-03-23T23:43:14.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.638+0000 7f82d2400700 1 -- start start 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.638+0000 7f82d2400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.638+0000 7f82d2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d2400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 0x7f82cc1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc10ff50 con 0x7f82cc0761c0 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d2400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc10fdd0 con 0x7f82cc109fd0 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d2400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1100d0 con 0x7f82cc074bf0 2024-03-23T23:43:14.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82cbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82cbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54024/0 (socket says 172.21.15.99:54024) 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82cbe00700 1 -- 172.21.15.99:0/3434127488 learned_addr learned my addr 172.21.15.99:0/3434127488 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1a00700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 0x7f82cc1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 -- 172.21.15.99:0/3434127488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 msgr2=0x7f82cc1a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:14.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 0x7f82cc1a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 -- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 msgr2=0x7f82cc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.639+0000 7f82d1000700 1 -- 172.21.15.99:0/3434127488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc1a21d0 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82d1000700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc19a570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f82c000e9f0 tx=0x7f82c000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c000c9c0 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82c0010430 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc1a24c0 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0018680 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.640+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc1a2a50 con 0x7f82cc074bf0 2024-03-23T23:43:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.641+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc068e50 con 0x7f82cc074bf0 2024-03-23T23:43:14.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.645+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f82c0010950 con 0x7f82cc074bf0 2024-03-23T23:43:14.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.646+0000 7f82c9600700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 0x7f82b40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:14.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.646+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82c00cead0 con 0x7f82cc074bf0 2024-03-23T23:43:14.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.646+0000 7f82cbe00700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 0x7f82b40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.647+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82c009bb50 con 0x7f82cc074bf0 2024-03-23T23:43:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.647+0000 7f82cbe00700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 0x7f82b40a4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b2e0 tx=0x7f82b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:14.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.970+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f82cc0625d0 con 0x7f82b40a2910 2024-03-23T23:43:14.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.973+0000 7f82c9600700 1 -- 172.21.15.99:0/3434127488 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f82cc0625d0 con 0x7f82b40a2910 2024-03-23T23:43:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:HOST ADDR LABELS STATUS 2024-03-23T23:43:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 172.21.15.99 2024-03-23T23:43:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:smithi149 172.21.15.149 2024-03-23T23:43:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:smithi174 172.21.15.174 2024-03-23T23:43:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:3 hosts in cluster 2024-03-23T23:43:14.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 msgr2=0x7f82b40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:14.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 0x7f82b40a4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b2e0 tx=0x7f82b8005fb0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 msgr2=0x7f82cc19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:14.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc19a570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f82c000e9f0 tx=0x7f82c000ed00 comp rx=0 tx=0).stop 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 shutdown_connections 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f82b40a2910 0x7f82b40a4dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82cc109fd0 0x7f82cc1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc074bf0 0x7f82cc19a570 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 --2- 172.21.15.99:0/3434127488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82cc0761c0 0x7f82cc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 >> 172.21.15.99:0/3434127488 conn(0x7f82cc0fd6a0 msgr2=0x7f82cc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 shutdown_connections 2024-03-23T23:43:14.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:14.977+0000 7f82d2400700 1 -- 172.21.15.99:0/3434127488 wait complete. 2024-03-23T23:43:15.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-23T23:43:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:15 smithi099 ceph-mon[31715]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:15 smithi099 ceph-mon[31715]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:15 smithi149 ceph-mon[36083]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:15 smithi149 ceph-mon[36083]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:15 smithi174 ceph-mon[37002]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:15 smithi174 ceph-mon[37002]: from='client.25099 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:16.377 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:18.049 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:17 smithi099 ceph-mon[31715]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:17 smithi149 ceph-mon[36083]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:18.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:17 smithi174 ceph-mon[37002]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 -- 172.21.15.99:0/961877812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 msgr2=0x7fb564100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 --2- 172.21.15.99:0/961877812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb564100d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb554009990 tx=0x7fb554009ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 -- 172.21.15.99:0/961877812 shutdown_connections 2024-03-23T23:43:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 --2- 172.21.15.99:0/961877812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 0x7fb564102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 --2- 172.21.15.99:0/961877812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb564100d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 --2- 172.21.15.99:0/961877812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb564101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 -- 172.21.15.99:0/961877812 >> 172.21.15.99:0/961877812 conn(0x7fb5640fc5b0 msgr2=0x7fb5640fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.585+0000 7fb568a00700 1 -- 172.21.15.99:0/961877812 shutdown_connections 2024-03-23T23:43:18.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.586+0000 7fb568a00700 1 -- 172.21.15.99:0/961877812 wait complete. 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 Processor -- start 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 -- start start 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 0x7fb5641a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56410fef0 con 0x7fb564101c90 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56410fd70 con 0x7fb5641012d0 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.587+0000 7fb568a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564110070 con 0x7fb564100970 2024-03-23T23:43:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb563400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:46354/0 (socket says 172.21.15.99:46354) 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb563400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:46736/0 (socket says 172.21.15.99:46736) 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 -- 172.21.15.99:0/3778017184 learned_addr learned my addr 172.21.15.99:0/3778017184 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb563e00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 0x7fb5641a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 -- 172.21.15.99:0/3778017184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 msgr2=0x7fb56419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 -- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 msgr2=0x7fb5641a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 0x7fb5641a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 -- 172.21.15.99:0/3778017184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5641a2170 con 0x7fb5641012d0 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb563400700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:18.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.588+0000 7fb562a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e9a0 tx=0x7fb55000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.589+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55000c9c0 con 0x7fb5641012d0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.589+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb55000cb60 con 0x7fb5641012d0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.589+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5641a2460 con 0x7fb5641012d0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.589+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550018680 con 0x7fb5641012d0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.589+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5641a29a0 con 0x7fb5641012d0 2024-03-23T23:43:18.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.590+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb564068e50 con 0x7fb5641012d0 2024-03-23T23:43:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.592+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb55000cd10 con 0x7fb5641012d0 2024-03-23T23:43:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.593+0000 7fb55be00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 0x7fb5480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.593+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb5500cec20 con 0x7fb5641012d0 2024-03-23T23:43:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.593+0000 7fb563400700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 0x7fb5480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:18.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.594+0000 7fb563400700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 0x7fb5480a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b2e0 tx=0x7fb554005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:18.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.597+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb55009bc40 con 0x7fb5641012d0 2024-03-23T23:43:18.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.882+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5640625d0 con 0x7fb5480a2890 2024-03-23T23:43:18.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.885+0000 7fb55be00700 1 -- 172.21.15.99:0/3778017184 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fb5640625d0 con 0x7fb5480a2890 2024-03-23T23:43:18.885 INFO:teuthology.orchestra.run.smithi099.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-23T23:43:18.885 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-23T23:43:18.885 INFO:teuthology.orchestra.run.smithi099.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-23T23:43:18.885 INFO:teuthology.orchestra.run.smithi099.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-23T23:43:18.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.887+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 msgr2=0x7fb5480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:18.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.887+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 0x7fb5480a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b2e0 tx=0x7fb554005fd0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.887+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 msgr2=0x7fb56419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.887+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e9a0 tx=0x7fb55000ecb0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 shutdown_connections 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb5480a2890 0x7fb5480a4d50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb564101c90 0x7fb5641a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb564100970 0x7fb56419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 --2- 172.21.15.99:0/3778017184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5641012d0 0x7fb56419aa50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 >> 172.21.15.99:0/3778017184 conn(0x7fb5640fc5b0 msgr2=0x7fb5640fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 shutdown_connections 2024-03-23T23:43:18.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:18.888+0000 7fb568a00700 1 -- 172.21.15.99:0/3778017184 wait complete. 2024-03-23T23:43:19.709 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T23:43:19.718 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:43:19.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T23:43:20.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:19 smithi099 ceph-mon[31715]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:20.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:19 smithi099 ceph-mon[31715]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:20.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:19 smithi174 ceph-mon[37002]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:19 smithi174 ceph-mon[37002]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:20.173 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:19 smithi149 ceph-mon[36083]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:19 smithi149 ceph-mon[36083]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T23:43:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:21 smithi174 ceph-mon[37002]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:22.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:21 smithi099 ceph-mon[31715]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:21 smithi149 ceph-mon[36083]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 -- 172.21.15.99:0/1148020046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4102cd0 msgr2=0x7ff2f41050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 --2- 172.21.15.99:0/1148020046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4102cd0 0x7ff2f41050c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009990 tx=0x7ff2e8009ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 -- 172.21.15.99:0/1148020046 shutdown_connections 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 --2- 172.21.15.99:0/1148020046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4107f30 0x7ff2f410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 --2- 172.21.15.99:0/1148020046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4102cd0 0x7ff2f41050c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 --2- 172.21.15.99:0/1148020046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4105600 0x7ff2f41079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 -- 172.21.15.99:0/1148020046 >> 172.21.15.99:0/1148020046 conn(0x7ff2f40fc660 msgr2=0x7ff2f40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 -- 172.21.15.99:0/1148020046 shutdown_connections 2024-03-23T23:43:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.437+0000 7ff2fba00700 1 -- 172.21.15.99:0/1148020046 wait complete. 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.438+0000 7ff2fba00700 1 Processor -- start 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.438+0000 7ff2fba00700 1 -- start start 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 0x7ff2f419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4107f30 0x7ff2f41a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410de20 con 0x7ff2f4102cd0 2024-03-23T23:43:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410dca0 con 0x7ff2f4105600 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2fba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410dfa0 con 0x7ff2f4107f30 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 0x7ff2f419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:52130/0 (socket says 172.21.15.99:52130) 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 -- 172.21.15.99:0/976996853 learned_addr learned my addr 172.21.15.99:0/976996853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f9a00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4107f30 0x7ff2f41a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 -- 172.21.15.99:0/976996853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4107f30 msgr2=0x7ff2f41a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4107f30 0x7ff2f41a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 -- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 msgr2=0x7ff2f419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.439+0000 7ff2f3e00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 0x7ff2f419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f3e00700 1 -- 172.21.15.99:0/976996853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f41a22e0 con 0x7ff2f4105600 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f9000700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 0x7ff2f419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f3e00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9c0 tx=0x7ff2e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000cbd0 con 0x7ff2f4105600 2024-03-23T23:43:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e000ee60 con 0x7ff2f4105600 2024-03-23T23:43:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f41a25d0 con 0x7ff2f4105600 2024-03-23T23:43:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f41a2b10 con 0x7ff2f4105600 2024-03-23T23:43:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.440+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0005680 con 0x7ff2f4105600 2024-03-23T23:43:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.442+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4005360 con 0x7ff2f4105600 2024-03-23T23:43:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.445+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff2e0004830 con 0x7ff2f4105600 2024-03-23T23:43:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.446+0000 7ff2f1600700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 0x7ff2dc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.446+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2e00cebd0 con 0x7ff2f4105600 2024-03-23T23:43:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.446+0000 7ff2f9000700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 0x7ff2dc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.447+0000 7ff2f9000700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 0x7ff2dc0a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b7b0 tx=0x7ff2e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.450+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff2e009bbf0 con 0x7ff2f4105600 2024-03-23T23:43:22.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.782+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2d4005e60 con 0x7ff2f4105600 2024-03-23T23:43:22.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.783+0000 7ff2f1600700 1 -- 172.21.15.99:0/976996853 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+3038 (secure 0 0 0) 0x7ff2e009b4f0 con 0x7ff2f4105600 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:e26 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:legacy client fscid: 1 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:Filesystem 'cephfs' (1) 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:fs_name cephfs 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 26 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-03-23T23:41:09.728597+0000 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-03-23T23:41:17.278370+0000 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:tableserver 0 2024-03-23T23:43:22.784 INFO:teuthology.orchestra.run.smithi099.stdout:root 0 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:session_timeout 300 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:session_autoclose 300 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:max_file_size 1099511627776 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:max_xattr_size 65536 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:required_client_features {} 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure 0 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure_osd_epoch 0 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:max_mds 5 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:in 0,1,2,3,4 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:up {0=24569,1=24565,2=24557,3=24547,4=24526} 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:failed 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:damaged 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:stopped 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:data_pools [3] 2024-03-23T23:43:22.785 INFO:teuthology.orchestra.run.smithi099.stdout:metadata_pool 2 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:inline_data disabled 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:balancer 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:bal_rank_mask -1 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:standby_count_wanted 0 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:qdb_cluster leader: 24569 members: 24547,24526,24557,24569,24565 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.h{0:24515} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/3655333447,v1:172.21.15.149:6837/3655333447] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.g{1:24565} state up:active seq 6 addr [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.f{1:24514} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/731827456,v1:172.21.15.174:6835/731827456] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a{2:24557} state up:active seq 5 addr [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.e{2:24485} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/1177424904,v1:172.21.15.149:6835/1177424904] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.i{3:24547} state up:active seq 7 addr [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.l{3:24448} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/4085463188,v1:172.21.15.174:6833/4085463188] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.786 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c{4:24526} state up:active seq 9 addr [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.k{4:24440} state up:standby-replay seq 1 addr [v2:172.21.15.149:6832/2919310963,v1:172.21.15.149:6833/2919310963] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout:Standby daemons: 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.j{-1:14391} state up:standby seq 1 addr [v2:172.21.15.99:6834/3136510370,v1:172.21.15.99:6835/3136510370] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.787 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.99:6836/2449593351,v1:172.21.15.99:6837/2449593351] compat {c=[1],r=[1],i=[fff]}] 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 msgr2=0x7ff2dc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 0x7ff2dc0a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b7b0 tx=0x7ff2e8005fd0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 msgr2=0x7ff2f419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9c0 tx=0x7ff2e000ecd0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 shutdown_connections 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2dc0a2910 0x7ff2dc0a4dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4107f30 0x7ff2f41a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2f4102cd0 0x7ff2f419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 --2- 172.21.15.99:0/976996853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f4105600 0x7ff2f419abc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 >> 172.21.15.99:0/976996853 conn(0x7ff2f40fc660 msgr2=0x7ff2f4102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 shutdown_connections 2024-03-23T23:43:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:22.787+0000 7ff2fba00700 1 -- 172.21.15.99:0/976996853 wait complete. 2024-03-23T23:43:22.789 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:43:23.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:22 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/976996853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T23:43:23.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:22 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/976996853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T23:43:23.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:22 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/976996853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T23:43:23.624 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-23T23:43:24.075 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:24.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:23 smithi174 ceph-mon[37002]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:24.231 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:23 smithi099 ceph-mon[31715]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:24.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:23 smithi149 ceph-mon[36083]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:25 smithi099 ceph-mon[31715]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:25 smithi174 ceph-mon[37002]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 -- 172.21.15.99:0/923423616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181021e0 msgr2=0x7f1818102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 --2- 172.21.15.99:0/923423616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181021e0 0x7f1818102600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18080099c0 tx=0x7f1808009cd0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 -- 172.21.15.99:0/923423616 shutdown_connections 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 --2- 172.21.15.99:0/923423616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 0x7f181810a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 --2- 172.21.15.99:0/923423616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181021e0 0x7f1818102600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 --2- 172.21.15.99:0/923423616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181033e0 0x7f1818103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.326+0000 7f181da00700 1 -- 172.21.15.99:0/923423616 >> 172.21.15.99:0/923423616 conn(0x7f18180fd740 msgr2=0x7f18180ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.327+0000 7f181da00700 1 -- 172.21.15.99:0/923423616 shutdown_connections 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.327+0000 7f181da00700 1 -- 172.21.15.99:0/923423616 wait complete. 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.328+0000 7f181da00700 1 Processor -- start 2024-03-23T23:43:26.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.328+0000 7f181da00700 1 -- start start 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 0x7f181819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 0x7f18181a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181810ddf0 con 0x7f1818103d80 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181810dc70 con 0x7f18181033e0 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f181da00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181810df70 con 0x7f18181021e0 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f1816a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:26.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f1816a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:51246/0 (socket says 172.21.15.99:51246) 2024-03-23T23:43:26.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f1816a00700 1 -- 172.21.15.99:0/3341705334 learned_addr learned my addr 172.21.15.99:0/3341705334 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f1817400700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 0x7f18181a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.329+0000 7f1816000700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 0x7f181819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 -- 172.21.15.99:0/3341705334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 msgr2=0x7f181819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 0x7f181819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 -- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 msgr2=0x7f18181a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 0x7f18181a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 -- 172.21.15.99:0/3341705334 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18181a22d0 con 0x7f18181021e0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816a00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f180800bee0 tx=0x7f180800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.330+0000 7f1816000700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 0x7f181819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.331+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808009e60 con 0x7f18181021e0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.331+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1808003e30 con 0x7f18181021e0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.331+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18181a2560 con 0x7f18181021e0 2024-03-23T23:43:26.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.331+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080173f0 con 0x7f18181021e0 2024-03-23T23:43:26.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.331+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18181a2a10 con 0x7f18181021e0 2024-03-23T23:43:26.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.332+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818068e50 con 0x7f18181021e0 2024-03-23T23:43:26.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.335+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f180800f5f0 con 0x7f18181021e0 2024-03-23T23:43:26.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.337+0000 7f180f400700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 0x7f17fc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:26.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.337+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f18080cf7c0 con 0x7f18181021e0 2024-03-23T23:43:26.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.337+0000 7f1816000700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 0x7f17fc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:26.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.338+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1808013070 con 0x7f18181021e0 2024-03-23T23:43:26.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.338+0000 7f1816000700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 0x7f17fc0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1804009680 tx=0x7f1804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:25 smithi149 ceph-mon[36083]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:26.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.573+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f18181a2e50 con 0x7f18181021e0 2024-03-23T23:43:26.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.574+0000 7f180f400700 1 -- 172.21.15.99:0/3341705334 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5340 (secure 0 0 0) 0x7f180809c870 con 0x7f18181021e0 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 87 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-03-23T23:31:55.387369+0000 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-03-23T23:42:07.872220+0000 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:crush_version 26 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:full_ratio 0.95 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:backfillfull_ratio 0.9 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:nearfull_ratio 0.85 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:require_min_compat_client luminous 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:min_compat_client luminous 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:require_osd_release squid 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:stretch_mode_enabled false 2024-03-23T23:43:26.575 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:max_osd 12 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6802/1797931451,v1:172.21.15.99:6803/1797931451] [v2:172.21.15.99:6804/1797931451,v1:172.21.15.99:6805/1797931451] exists,up 819aae5f-09d9-444c-953a-ec0173dc128e 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/809430535,v1:172.21.15.149:6801/809430535] [v2:172.21.15.149:6802/809430535,v1:172.21.15.149:6803/809430535] exists,up 1af37efa-acb5-4b3c-bfbd-a43df66c3893 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/3172830280,v1:172.21.15.174:6801/3172830280] [v2:172.21.15.174:6802/3172830280,v1:172.21.15.174:6803/3172830280] exists,up 6afadce7-6cd9-439a-8916-36eb8eb44c99 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/1963358194,v1:172.21.15.99:6811/1963358194] [v2:172.21.15.99:6812/1963358194,v1:172.21.15.99:6813/1963358194] exists,up acb0cde2-7e55-4edf-baa5-3f6c46a83299 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/766703080,v1:172.21.15.149:6809/766703080] [v2:172.21.15.149:6810/766703080,v1:172.21.15.149:6811/766703080] exists,up 2d32abb2-0d09-4f68-a5a0-2a823ec6a012 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/1509311749,v1:172.21.15.174:6809/1509311749] [v2:172.21.15.174:6810/1509311749,v1:172.21.15.174:6811/1509311749] exists,up 8f4ee3c4-a445-4abf-bf98-25a56828b679 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/1423285526,v1:172.21.15.99:6819/1423285526] [v2:172.21.15.99:6820/1423285526,v1:172.21.15.99:6821/1423285526] exists,up 0a94e089-ed73-4d72-b458-74bf3589a1fd 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/1916985722,v1:172.21.15.149:6817/1916985722] [v2:172.21.15.149:6818/1916985722,v1:172.21.15.149:6819/1916985722] exists,up db44c8d6-8d15-4514-a5e3-0babe7b5b17b 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/2258533279,v1:172.21.15.174:6817/2258533279] [v2:172.21.15.174:6818/2258533279,v1:172.21.15.174:6819/2258533279] exists,up 0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6826/4103401767,v1:172.21.15.99:6827/4103401767] [v2:172.21.15.99:6828/4103401767,v1:172.21.15.99:6829/4103401767] exists,up 3720287f-1023-43eb-bce3-280e52715f41 2024-03-23T23:43:26.576 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/2851714009,v1:172.21.15.149:6825/2851714009] [v2:172.21.15.149:6826/2851714009,v1:172.21.15.149:6827/2851714009] exists,up b134a1cf-77b1-4218-b117-38efbd289712 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/2854369020,v1:172.21.15.174:6825/2854369020] [v2:172.21.15.174:6826/2854369020,v1:172.21.15.174:6827/2854369020] exists,up b1fa75b5-9238-4a2e-bce2-4ab94569fbfd 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.0 [7,11] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.b [2,4] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.1b [2,1] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 2.1d [8,11] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.3 [4,9] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.19 [3,7] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1b [4,6] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1e [3,9] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.23 [4,8] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.33 [5,9] 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/3702331558 expires 2024-03-24T23:33:06.585425+0000 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/2749486918 expires 2024-03-24T23:33:06.585425+0000 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/1079793222 expires 2024-03-24T23:32:29.874809+0000 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2100135742 expires 2024-03-24T23:32:29.874809+0000 2024-03-23T23:43:26.577 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/3938208331 expires 2024-03-24T23:32:29.874809+0000 2024-03-23T23:43:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2517072551 expires 2024-03-24T23:33:06.585425+0000 2024-03-23T23:43:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/2749486918 expires 2024-03-24T23:33:06.585425+0000 2024-03-23T23:43:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/686293385 expires 2024-03-24T23:33:06.585425+0000 2024-03-23T23:43:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/3938208331 expires 2024-03-24T23:32:29.874809+0000 2024-03-23T23:43:26.578 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2402511247 expires 2024-03-24T23:32:29.874809+0000 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 msgr2=0x7f17fc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 0x7f17fc0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1804009680 tx=0x7f1804009260 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 msgr2=0x7f181819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f180800bee0 tx=0x7f180800bf10 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 shutdown_connections 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f17fc0a2910 0x7f17fc0a4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1818103d80 0x7f18181a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18181021e0 0x7f181819a670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 --2- 172.21.15.99:0/3341705334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18181033e0 0x7f181819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.577+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 >> 172.21.15.99:0/3341705334 conn(0x7f18180fd740 msgr2=0x7f18180743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.578+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 shutdown_connections 2024-03-23T23:43:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:26.578+0000 7f181da00700 1 -- 172.21.15.99:0/3341705334 wait complete. 2024-03-23T23:43:27.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3341705334' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T23:43:27.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3341705334' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T23:43:27.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3341705334' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T23:43:27.443 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-23T23:43:27.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:27.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 -- 172.21.15.99:0/22330909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac1026f0 msgr2=0x7ff7ac102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:27.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/22330909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac102b70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff798009900 tx=0x7ff79802bc40 comp rx=0 tx=0).stop 2024-03-23T23:43:27.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 -- 172.21.15.99:0/22330909 shutdown_connections 2024-03-23T23:43:27.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/22330909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/22330909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac102b70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/22330909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac101500 0x7ff7ac101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 -- 172.21.15.99:0/22330909 >> 172.21.15.99:0/22330909 conn(0x7ff7ac0fca70 msgr2=0x7ff7ac0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 -- 172.21.15.99:0/22330909 shutdown_connections 2024-03-23T23:43:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.688+0000 7ff7b0a00640 1 -- 172.21.15.99:0/22330909 wait complete. 2024-03-23T23:43:27.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 Processor -- start 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 -- start start 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac10d1e0 con 0x7ff7ac1030b0 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac10d060 con 0x7ff7ac101500 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.690+0000 7ff7b0a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac10d360 con 0x7ff7ac1026f0 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:52178/0 (socket says 172.21.15.99:52178) 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 -- 172.21.15.99:0/2531115121 learned_addr learned my addr 172.21.15.99:0/2531115121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7a9e00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:27.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7ab200640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 -- 172.21.15.99:0/2531115121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1026f0 msgr2=0x7ff7ac073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 -- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 msgr2=0x7ff7ac073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 -- 172.21.15.99:0/2531115121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac074860 con 0x7ff7ac101500 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7ab200640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.691+0000 7ff7aa800640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00d770 tx=0x7ff79c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.692+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c017070 con 0x7ff7ac101500 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.692+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff79c009c30 con 0x7ff7ac101500 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.692+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac1a1970 con 0x7ff7ac101500 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.692+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c003dd0 con 0x7ff7ac101500 2024-03-23T23:43:27.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.692+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac1a1e90 con 0x7ff7ac101500 2024-03-23T23:43:27.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.693+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac0683b0 con 0x7ff7ac101500 2024-03-23T23:43:27.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.695+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff79c003f70 con 0x7ff7ac101500 2024-03-23T23:43:27.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.696+0000 7ff7a3400640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 0x7ff7900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:27.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.697+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff79c0ce460 con 0x7ff7ac101500 2024-03-23T23:43:27.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.697+0000 7ff7a9e00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 0x7ff7900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:27.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.698+0000 7ff7a9e00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 0x7ff7900a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff79802efd0 tx=0x7ff79802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:27.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.701+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff79c09b480 con 0x7ff7ac101500 2024-03-23T23:43:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.924+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7ac0622a0 con 0x7ff7ac101500 2024-03-23T23:43:27.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.926+0000 7ff7a3400640 1 -- 172.21.15.99:0/2531115121 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7ff79c009de0 con 0x7ff7ac101500 2024-03-23T23:43:27.926 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:27.926 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.928+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 msgr2=0x7ff7900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.928+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 0x7ff7900a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff79802efd0 tx=0x7ff79802cf90 comp rx=0 tx=0).stop 2024-03-23T23:43:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 msgr2=0x7ff7ac075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00d770 tx=0x7ff79c00dc30 comp rx=0 tx=0).stop 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 shutdown_connections 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7900a2800 0x7ff7900a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac1030b0 0x7ff7ac073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7ac1026f0 0x7ff7ac073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 --2- 172.21.15.99:0/2531115121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7ac101500 0x7ff7ac075450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 >> 172.21.15.99:0/2531115121 conn(0x7ff7ac0fca70 msgr2=0x7ff7ac105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 shutdown_connections 2024-03-23T23:43:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:27.929+0000 7ff7b0a00640 1 -- 172.21.15.99:0/2531115121 wait complete. 2024-03-23T23:43:27.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:43:27.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:43:28.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 -- 172.21.15.99:0/2700406672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec0ffe70 msgr2=0x7fbcec100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/2700406672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec0ffe70 0x7fbcec100290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009900 tx=0x7fbce002bcc0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 -- 172.21.15.99:0/2700406672 shutdown_connections 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/2700406672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec101170 0x7fbcec1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/2700406672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec1007d0 0x7fbcec100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/2700406672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec0ffe70 0x7fbcec100290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 -- 172.21.15.99:0/2700406672 >> 172.21.15.99:0/2700406672 conn(0x7fbcec0fba10 msgr2=0x7fbcec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 -- 172.21.15.99:0/2700406672 shutdown_connections 2024-03-23T23:43:28.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.139+0000 7fbcf2c00640 1 -- 172.21.15.99:0/2700406672 wait complete. 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.140+0000 7fbcf2c00640 1 Processor -- start 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.140+0000 7fbcf2c00640 1 -- start start 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec1007d0 0x7fbcec199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 0x7fbcec1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10f420 con 0x7fbcec1007d0 2024-03-23T23:43:28.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10f2a0 con 0x7fbcec101170 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10f5a0 con 0x7fbcec0ffe70 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:51274/0 (socket says 172.21.15.99:51274) 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 -- 172.21.15.99:0/3146697716 learned_addr learned my addr 172.21.15.99:0/3146697716 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2200640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 0x7fbcec1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf0e00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec1007d0 0x7fbcec199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 -- 172.21.15.99:0/3146697716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 msgr2=0x7fbcec1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 0x7fbcec1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 -- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec1007d0 msgr2=0x7fbcec199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec1007d0 0x7fbcec199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf1800640 1 -- 172.21.15.99:0/3146697716 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec1a1690 con 0x7fbcec0ffe70 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.141+0000 7fbcf2200640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 0x7fbcec1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcf1800640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce002c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:28.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0043020 con 0x7fbcec0ffe70 2024-03-23T23:43:28.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbce00054e0 con 0x7fbcec0ffe70 2024-03-23T23:43:28.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec1a1920 con 0x7fbcec0ffe70 2024-03-23T23:43:28.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0003c20 con 0x7fbcec0ffe70 2024-03-23T23:43:28.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.142+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec1a1ec0 con 0x7fbcec0ffe70 2024-03-23T23:43:28.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.144+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec10abc0 con 0x7fbcec0ffe70 2024-03-23T23:43:28.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.146+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbce0046020 con 0x7fbcec0ffe70 2024-03-23T23:43:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.149+0000 7fbcea000640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 0x7fbcd40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.149+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbce00f0a80 con 0x7fbcec0ffe70 2024-03-23T23:43:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.149+0000 7fbcf0e00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 0x7fbcd40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.150+0000 7fbcf0e00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 0x7fbcd40a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcec19af80 tx=0x7fbcd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.152+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbce00f2e20 con 0x7fbcec0ffe70 2024-03-23T23:43:28.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.349+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcec0622a0 con 0x7fbcec0ffe70 2024-03-23T23:43:28.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.350+0000 7fbcea000640 1 -- 172.21.15.99:0/3146697716 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbce004d020 con 0x7fbcec0ffe70 2024-03-23T23:43:28.350 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.352+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 msgr2=0x7fbcd40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.352+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 0x7fbcd40a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcec19af80 tx=0x7fbcd8005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.352+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 msgr2=0x7fbcec199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.352+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce002c6e0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 shutdown_connections 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbcd40a2880 0x7fbcd40a4d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101170 0x7fbcec1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcec1007d0 0x7fbcec199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 --2- 172.21.15.99:0/3146697716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec0ffe70 0x7fbcec199a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 >> 172.21.15.99:0/3146697716 conn(0x7fbcec0fba10 msgr2=0x7fbcec0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 shutdown_connections 2024-03-23T23:43:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.353+0000 7fbcf2c00640 1 -- 172.21.15.99:0/3146697716 wait complete. 2024-03-23T23:43:28.360 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:27 smithi099 ceph-mon[31715]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:28.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:43:28.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:27 smithi149 ceph-mon[36083]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:28.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:27 smithi174 ceph-mon[37002]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 -- 172.21.15.99:0/1820725901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8102770 msgr2=0x7ff9f8102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 --2- 172.21.15.99:0/1820725901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8102770 0x7ff9f8102bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009900 tx=0x7ff9e402bc40 comp rx=0 tx=0).stop 2024-03-23T23:43:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 -- 172.21.15.99:0/1820725901 shutdown_connections 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 --2- 172.21.15.99:0/1820725901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8103130 0x7ff9f81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 --2- 172.21.15.99:0/1820725901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8102770 0x7ff9f8102bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 --2- 172.21.15.99:0/1820725901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8101580 0x7ff9f81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 -- 172.21.15.99:0/1820725901 >> 172.21.15.99:0/1820725901 conn(0x7ff9f80fcb10 msgr2=0x7ff9f80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 -- 172.21.15.99:0/1820725901 shutdown_connections 2024-03-23T23:43:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.600+0000 7ff9fd200640 1 -- 172.21.15.99:0/1820725901 wait complete. 2024-03-23T23:43:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.601+0000 7ff9fd200640 1 Processor -- start 2024-03-23T23:43:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.601+0000 7ff9fd200640 1 -- start start 2024-03-23T23:43:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8102770 0x7ff9f8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8103130 0x7ff9f81a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f810d230 con 0x7ff9f8102770 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f810d0b0 con 0x7ff9f8103130 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9fd200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f810d3b0 con 0x7ff9f8101580 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8102770 0x7ff9f8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:51288/0 (socket says 172.21.15.99:51288) 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 -- 172.21.15.99:0/3168553422 learned_addr learned my addr 172.21.15.99:0/3168553422 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7e00640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8103130 0x7ff9f81a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 -- 172.21.15.99:0/3168553422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8103130 msgr2=0x7ff9f81a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8103130 0x7ff9f81a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 -- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8102770 msgr2=0x7ff9f8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8102770 0x7ff9f8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.602+0000 7ff9f7400640 1 -- 172.21.15.99:0/3168553422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f81a1670 con 0x7ff9f8101580 2024-03-23T23:43:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9f7400640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800e9c0 tx=0x7ff9e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800cd80 con 0x7ff9f8101580 2024-03-23T23:43:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9e80035f0 con 0x7ff9f8101580 2024-03-23T23:43:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81a1960 con 0x7ff9f8101580 2024-03-23T23:43:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8010690 con 0x7ff9f8101580 2024-03-23T23:43:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.603+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f81a1e80 con 0x7ff9f8101580 2024-03-23T23:43:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.604+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8068800 con 0x7ff9f8101580 2024-03-23T23:43:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.608+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff9e80037a0 con 0x7ff9f8101580 2024-03-23T23:43:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.609+0000 7ff9efe00640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 0x7ff9dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.609+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff9e80cee30 con 0x7ff9f8101580 2024-03-23T23:43:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.609+0000 7ff9f6a00640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 0x7ff9dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.610+0000 7ff9f6a00640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 0x7ff9dc0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402c250 tx=0x7ff9e402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.613+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9e809beb0 con 0x7ff9f8101580 2024-03-23T23:43:28.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.813+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9f81a2810 con 0x7ff9f8101580 2024-03-23T23:43:28.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.814+0000 7ff9efe00640 1 -- 172.21.15.99:0/3168553422 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9e809b7b0 con 0x7ff9f8101580 2024-03-23T23:43:28.814 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 msgr2=0x7ff9dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 0x7ff9dc0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff9e402c250 tx=0x7ff9e402cf90 comp rx=0 tx=0).stop 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 msgr2=0x7ff9f8199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800e9c0 tx=0x7ff9e800ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 shutdown_connections 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff9dc0a2800 0x7ff9dc0a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f8103130 0x7ff9f81a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f8102770 0x7ff9f8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 --2- 172.21.15.99:0/3168553422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101580 0x7ff9f8199a50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:28.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 >> 172.21.15.99:0/3168553422 conn(0x7ff9f80fcb10 msgr2=0x7ff9f80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:28.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 shutdown_connections 2024-03-23T23:43:28.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:28.816+0000 7ff9fd200640 1 -- 172.21.15.99:0/3168553422 wait complete. 2024-03-23T23:43:28.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 -- 172.21.15.99:0/4058296294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 msgr2=0x7fd61c10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 --2- 172.21.15.99:0/4058296294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c10b8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd61402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 -- 172.21.15.99:0/4058296294 shutdown_connections 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 --2- 172.21.15.99:0/4058296294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c10b8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 --2- 172.21.15.99:0/4058296294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c074100 0x7fd61c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 --2- 172.21.15.99:0/4058296294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c0756e0 0x7fd61c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 -- 172.21.15.99:0/4058296294 >> 172.21.15.99:0/4058296294 conn(0x7fd61c0fcb10 msgr2=0x7fd61c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 -- 172.21.15.99:0/4058296294 shutdown_connections 2024-03-23T23:43:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.049+0000 7fd621c00640 1 -- 172.21.15.99:0/4058296294 wait complete. 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 Processor -- start 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 -- start start 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 0x7fd61c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c10f3e0 con 0x7fd61c074100 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c10f260 con 0x7fd61c0756e0 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.050+0000 7fd621c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c10f560 con 0x7fd61c109460 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 0x7fd61c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34000/0 (socket says 172.21.15.99:34000) 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 0x7fd61c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39122/0 (socket says 172.21.15.99:39122) 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 -- 172.21.15.99:0/3176387628 learned_addr learned my addr 172.21.15.99:0/3176387628 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd621200640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 -- 172.21.15.99:0/3176387628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 msgr2=0x7fd61c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 -- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 msgr2=0x7fd61c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 0x7fd61c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 -- 172.21.15.99:0/3176387628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c1a1610 con 0x7fd61c0756e0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd61b400640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e9f0 tx=0x7fd60800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.051+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800cd80 con 0x7fd61c0756e0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.052+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd608010910 con 0x7fd61c0756e0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.052+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c1a1900 con 0x7fd61c0756e0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.052+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608018990 con 0x7fd61c0756e0 2024-03-23T23:43:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.052+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c1a1e50 con 0x7fd61c0756e0 2024-03-23T23:43:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.053+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c108190 con 0x7fd61c0756e0 2024-03-23T23:43:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.055+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd608010ac0 con 0x7fd61c0756e0 2024-03-23T23:43:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.056+0000 7fd618c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 0x7fd6000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.056+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd6080cee30 con 0x7fd61c0756e0 2024-03-23T23:43:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.057+0000 7fd61be00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 0x7fd6000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.057+0000 7fd61be00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 0x7fd6000a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009bd0 tx=0x7fd60c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.060+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd60809b800 con 0x7fd61c0756e0 2024-03-23T23:43:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2531115121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3146697716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3168553422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.329+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd61c0622a0 con 0x7fd61c0756e0 2024-03-23T23:43:29.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.330+0000 7fd618c00640 1 -- 172.21.15.99:0/3176387628 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd60809f8a0 con 0x7fd61c0756e0 2024-03-23T23:43:29.330 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:43:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 msgr2=0x7fd6000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 0x7fd6000a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009bd0 tx=0x7fd60c005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 msgr2=0x7fd61c199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e9f0 tx=0x7fd60800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 shutdown_connections 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6000a2800 0x7fd6000a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c109460 0x7fd61c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd61c0756e0 0x7fd61c199f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 --2- 172.21.15.99:0/3176387628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd61c074100 0x7fd61c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 >> 172.21.15.99:0/3176387628 conn(0x7fd61c0fcb10 msgr2=0x7fd61c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.332+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 shutdown_connections 2024-03-23T23:43:29.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.333+0000 7fd621c00640 1 -- 172.21.15.99:0/3176387628 wait complete. 2024-03-23T23:43:29.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:43:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2531115121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3146697716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3168553422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2531115121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3146697716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3168553422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:29.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 -- 172.21.15.99:0/4097496357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01201038e0 msgr2=0x7f0120103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 --2- 172.21.15.99:0/4097496357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01201038e0 0x7f0120103d00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0110009ca0 tx=0x7f011002c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 -- 172.21.15.99:0/4097496357 shutdown_connections 2024-03-23T23:43:29.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 --2- 172.21.15.99:0/4097496357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 0x7f012010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 --2- 172.21.15.99:0/4097496357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0120104ad0 0x7f0120104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 --2- 172.21.15.99:0/4097496357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01201038e0 0x7f0120103d00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 -- 172.21.15.99:0/4097496357 >> 172.21.15.99:0/4097496357 conn(0x7f01200fee90 msgr2=0x7f01201012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:29.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 -- 172.21.15.99:0/4097496357 shutdown_connections 2024-03-23T23:43:29.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.541+0000 7f0126000640 1 -- 172.21.15.99:0/4097496357 wait complete. 2024-03-23T23:43:29.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 Processor -- start 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 -- start start 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0120104ad0 0x7f012019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 0x7f01201a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012010f550 con 0x7f0120105490 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012010f3d0 con 0x7f01201038e0 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.542+0000 7f0126000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012010f6d0 con 0x7f0120104ad0 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f0116a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0120104ad0 0x7f012019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34026/0 (socket says 172.21.15.99:34026) 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 -- 172.21.15.99:0/931597588 learned_addr learned my addr 172.21.15.99:0/931597588 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011fe00640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 0x7f01201a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 -- 172.21.15.99:0/931597588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0120104ad0 msgr2=0x7f012019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0120104ad0 0x7f012019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 -- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 msgr2=0x7f01201a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 0x7f01201a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 -- 172.21.15.99:0/931597588 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01201a1850 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011f400640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0110000f80 tx=0x7f0110032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.543+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110041020 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.544+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0110034cd0 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.544+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01201a1ae0 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.544+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110039040 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.544+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01201a1ff0 con 0x7f01201038e0 2024-03-23T23:43:29.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.545+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012010ac50 con 0x7f01201038e0 2024-03-23T23:43:29.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.547+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0110044020 con 0x7f01201038e0 2024-03-23T23:43:29.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.548+0000 7f011cc00640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 0x7f01040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:29.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.548+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01100f08e0 con 0x7f01201038e0 2024-03-23T23:43:29.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.548+0000 7f0116a00640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 0x7f01040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:29.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.549+0000 7f0116a00640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 0x7f01040a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0108009bd0 tx=0x7f0108005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:29.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.552+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f011004b020 con 0x7f01201038e0 2024-03-23T23:43:29.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.798+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012002d070 con 0x7f01201038e0 2024-03-23T23:43:29.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.798+0000 7f011cc00640 1 -- 172.21.15.99:0/931597588 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f01100348f0 con 0x7f01201038e0 2024-03-23T23:43:29.799 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:29.799 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:43:29.799 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 msgr2=0x7f01040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 0x7f01040a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0108009bd0 tx=0x7f0108005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 msgr2=0x7f0120199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0110000f80 tx=0x7f0110032ec0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 shutdown_connections 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f01040a2880 0x7f01040a4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120105490 0x7f01201a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0120104ad0 0x7f012019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.801+0000 7f0126000640 1 --2- 172.21.15.99:0/931597588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01201038e0 0x7f0120199c40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.802+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 >> 172.21.15.99:0/931597588 conn(0x7f01200fee90 msgr2=0x7f0120107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.802+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 shutdown_connections 2024-03-23T23:43:29.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:29.802+0000 7f0126000640 1 -- 172.21.15.99:0/931597588 wait complete. 2024-03-23T23:43:29.820 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 -- 172.21.15.99:0/1765721351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3864101500 msgr2=0x7f3864101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 --2- 172.21.15.99:0/1765721351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3864101500 0x7f3864101920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3854009900 tx=0x7f385402bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 -- 172.21.15.99:0/1765721351 shutdown_connections 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 --2- 172.21.15.99:0/1765721351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f3864109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 --2- 172.21.15.99:0/1765721351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38641026f0 0x7f3864102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 --2- 172.21.15.99:0/1765721351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3864101500 0x7f3864101920 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 -- 172.21.15.99:0/1765721351 >> 172.21.15.99:0/1765721351 conn(0x7f38640fca70 msgr2=0x7f38640feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 -- 172.21.15.99:0/1765721351 shutdown_connections 2024-03-23T23:43:30.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.018+0000 7f3863200640 1 -- 172.21.15.99:0/1765721351 wait complete. 2024-03-23T23:43:30.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 Processor -- start 2024-03-23T23:43:30.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 -- start start 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3864101500 0x7f3864197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f386419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386410d1b0 con 0x7f38641030b0 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386410d030 con 0x7f3864101500 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.020+0000 7f3863200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386410d330 con 0x7f38641026f0 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3862800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f386419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41782/0 (socket says 172.21.15.99:41782) 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 -- 172.21.15.99:0/1659899933 learned_addr learned my addr 172.21.15.99:0/1659899933 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:30.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3864101500 0x7f3864197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 -- 172.21.15.99:0/1659899933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3864101500 msgr2=0x7f3864197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3864101500 0x7f3864197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 -- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 msgr2=0x7f386419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f386419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 -- 172.21.15.99:0/1659899933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386419f440 con 0x7f38641026f0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3862800640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f386419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.021+0000 7f3861400640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f385000e9f0 tx=0x7f385000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.022+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385000cd80 con 0x7f38641026f0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.022+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3850010910 con 0x7f38641026f0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.022+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386419f730 con 0x7f38641026f0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.022+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850018990 con 0x7f38641026f0 2024-03-23T23:43:30.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.022+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386419fcd0 con 0x7f38641026f0 2024-03-23T23:43:30.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.023+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3864108870 con 0x7f38641026f0 2024-03-23T23:43:30.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.026+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3850010ac0 con 0x7f38641026f0 2024-03-23T23:43:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.030+0000 7f385aa00640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 0x7f38480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.030+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38500ceef0 con 0x7f38641026f0 2024-03-23T23:43:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.030+0000 7f3861e00640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 0x7f38480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.031+0000 7f3861e00640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 0x7f38480a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f385402c170 tx=0x7f385402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.033+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f385009b8c0 con 0x7f38641026f0 2024-03-23T23:43:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:29 smithi099 ceph-mon[31715]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:29 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3176387628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:30.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:29 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/931597588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.250+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f386402d240 con 0x7f38641026f0 2024-03-23T23:43:30.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.253+0000 7f385aa00640 1 -- 172.21.15.99:0/1659899933 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f385009f960 con 0x7f38641026f0 2024-03-23T23:43:30.253 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:30.253 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:30.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 msgr2=0x7f38480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 0x7f38480a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f385402c170 tx=0x7f385402df90 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 msgr2=0x7f3864197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f385000e9f0 tx=0x7f385000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 shutdown_connections 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38480a2880 0x7f38480a4d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38641030b0 0x7f386419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38641026f0 0x7f3864197d60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 --2- 172.21.15.99:0/1659899933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3864101500 0x7f3864197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.255+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 >> 172.21.15.99:0/1659899933 conn(0x7f38640fca70 msgr2=0x7f3864105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.256+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 shutdown_connections 2024-03-23T23:43:30.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.256+0000 7f3863200640 1 -- 172.21.15.99:0/1659899933 wait complete. 2024-03-23T23:43:30.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:43:30.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:43:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:29 smithi149 ceph-mon[36083]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:29 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3176387628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:29 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/931597588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:30.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:29 smithi174 ceph-mon[37002]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:30.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:29 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3176387628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:29 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/931597588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.476+0000 7f707ae00640 1 -- 172.21.15.99:0/2671505068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7074102150 msgr2=0x7f70741089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.476+0000 7f707ae00640 1 --2- 172.21.15.99:0/2671505068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7074102150 0x7f70741089e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f706c009be0 tx=0x7f706c02bef0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 -- 172.21.15.99:0/2671505068 shutdown_connections 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 --2- 172.21.15.99:0/2671505068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7074102150 0x7f70741089e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 --2- 172.21.15.99:0/2671505068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 --2- 172.21.15.99:0/2671505068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70741005a0 0x7f70741009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 -- 172.21.15.99:0/2671505068 >> 172.21.15.99:0/2671505068 conn(0x7f70740fbb30 msgr2=0x7f70740fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 -- 172.21.15.99:0/2671505068 shutdown_connections 2024-03-23T23:43:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.477+0000 7f707ae00640 1 -- 172.21.15.99:0/2671505068 wait complete. 2024-03-23T23:43:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.478+0000 7f707ae00640 1 Processor -- start 2024-03-23T23:43:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.478+0000 7f707ae00640 1 -- start start 2024-03-23T23:43:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7074102150 0x7f70741030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707410c200 con 0x7f7074102150 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707410c080 con 0x7f7074101790 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f707ae00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707410c380 con 0x7f70741005a0 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41796/0 (socket says 172.21.15.99:41796) 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 -- 172.21.15.99:0/2877940778 learned_addr learned my addr 172.21.15.99:0/2877940778 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073400640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7078e00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7074102150 0x7f70741030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 -- 172.21.15.99:0/2877940778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 msgr2=0x7f7074102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 -- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7074102150 msgr2=0x7f70741030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7074102150 0x7f70741030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073e00640 1 -- 172.21.15.99:0/2877940778 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074103940 con 0x7f70741005a0 2024-03-23T23:43:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.479+0000 7f7073400640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f7073e00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9c0 tx=0x7f706400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400cd80 con 0x7f70741005a0 2024-03-23T23:43:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70640035f0 con 0x7f70741005a0 2024-03-23T23:43:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70741a1ab0 con 0x7f70741005a0 2024-03-23T23:43:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064010690 con 0x7f70741005a0 2024-03-23T23:43:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.480+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70741a2050 con 0x7f70741005a0 2024-03-23T23:43:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.481+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70741a2220 con 0x7f70741005a0 2024-03-23T23:43:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.484+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f70640037a0 con 0x7f70741005a0 2024-03-23T23:43:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.486+0000 7f7070c00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 0x7f70580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.486+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f70640cee10 con 0x7f70741005a0 2024-03-23T23:43:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.486+0000 7f7073400640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 0x7f70580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.487+0000 7f7073400640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 0x7f70580a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70600096c0 tx=0x7f7060009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.493+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f706409be90 con 0x7f70741005a0 2024-03-23T23:43:30.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.692+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70741a23f0 con 0x7f70741005a0 2024-03-23T23:43:30.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.692+0000 7f7070c00640 1 -- 172.21.15.99:0/2877940778 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f706409b790 con 0x7f70741005a0 2024-03-23T23:43:30.692 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:43:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 msgr2=0x7f70580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 0x7f70580a4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f70600096c0 tx=0x7f7060009260 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 msgr2=0x7f7074104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9c0 tx=0x7f706400ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 shutdown_connections 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f70580a2880 0x7f70580a4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7074102150 0x7f70741030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7074101790 0x7f7074102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 --2- 172.21.15.99:0/2877940778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70741005a0 0x7f7074104530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 >> 172.21.15.99:0/2877940778 conn(0x7f70740fbb30 msgr2=0x7f70740fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 shutdown_connections 2024-03-23T23:43:30.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.695+0000 7f707ae00640 1 -- 172.21.15.99:0/2877940778 wait complete. 2024-03-23T23:43:30.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:43:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 -- 172.21.15.99:0/2528647234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27781038e0 msgr2=0x7f2778103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 --2- 172.21.15.99:0/2528647234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27781038e0 0x7f2778103d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2768009900 tx=0x7f276802bc40 comp rx=0 tx=0).stop 2024-03-23T23:43:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 -- 172.21.15.99:0/2528647234 shutdown_connections 2024-03-23T23:43:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 --2- 172.21.15.99:0/2528647234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 0x7f277810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 --2- 172.21.15.99:0/2528647234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778104ad0 0x7f2778104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 --2- 172.21.15.99:0/2528647234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27781038e0 0x7f2778103d00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 -- 172.21.15.99:0/2528647234 >> 172.21.15.99:0/2528647234 conn(0x7f27780fee50 msgr2=0x7f2778101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 -- 172.21.15.99:0/2528647234 shutdown_connections 2024-03-23T23:43:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.917+0000 7f277cc00640 1 -- 172.21.15.99:0/2528647234 wait complete. 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.918+0000 7f277cc00640 1 Processor -- start 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.918+0000 7f277cc00640 1 -- start start 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 0x7f27781a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277810f580 con 0x7f2778105490 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277810f400 con 0x7f27781038e0 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f277cc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277810f700 con 0x7f2778104ad0 2024-03-23T23:43:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2775400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2775e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2775e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34078/0 (socket says 172.21.15.99:34078) 2024-03-23T23:43:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2775400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41804/0 (socket says 172.21.15.99:41804) 2024-03-23T23:43:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2775e00640 1 -- 172.21.15.99:0/1401213839 learned_addr learned my addr 172.21.15.99:0/1401213839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.919+0000 7f2776800640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 0x7f27781a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 -- 172.21.15.99:0/1401213839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 msgr2=0x7f277819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 -- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 msgr2=0x7f27781a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 0x7f27781a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 -- 172.21.15.99:0/1401213839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27781a17f0 con 0x7f27781038e0 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775400640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f2775e00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f276802efd0 tx=0x7f276802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768041070 con 0x7f27781038e0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2768004450 con 0x7f27781038e0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27781a1a80 con 0x7f27781038e0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.920+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768003e50 con 0x7f27781038e0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.921+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27781a1f90 con 0x7f27781038e0 2024-03-23T23:43:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.922+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277810ac00 con 0x7f27781038e0 2024-03-23T23:43:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.925+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2768044070 con 0x7f27781038e0 2024-03-23T23:43:30.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.926+0000 7f276ea00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 0x7f275c0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:30.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.926+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f27680f0590 con 0x7f27781038e0 2024-03-23T23:43:30.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.926+0000 7f2775400640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 0x7f275c0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:30.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.927+0000 7f2775400640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 0x7f275c0a4ee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f277819b0e0 tx=0x7f2764005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:30.930+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f27680bcfc0 con 0x7f27781038e0 2024-03-23T23:43:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:30 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1659899933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:31.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:30 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2877940778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.141+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27780622a0 con 0x7f27781038e0 2024-03-23T23:43:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.142+0000 7f276ea00640 1 -- 172.21.15.99:0/1401213839 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27680c1060 con 0x7f27781038e0 2024-03-23T23:43:31.142 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:43:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 msgr2=0x7f275c0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 0x7f275c0a4ee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f277819b0e0 tx=0x7f2764005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 msgr2=0x7f2778199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f276802efd0 tx=0x7f276802c660 comp rx=0 tx=0).stop 2024-03-23T23:43:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 shutdown_connections 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f275c0a2a20 0x7f275c0a4ee0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2778105490 0x7f27781a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2778104ad0 0x7f277819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 --2- 172.21.15.99:0/1401213839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27781038e0 0x7f2778199bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 >> 172.21.15.99:0/1401213839 conn(0x7f27780fee50 msgr2=0x7f2778107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 shutdown_connections 2024-03-23T23:43:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.145+0000 7f277cc00640 1 -- 172.21.15.99:0/1401213839 wait complete. 2024-03-23T23:43:31.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 -- 172.21.15.99:0/3611328858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c1049b0 msgr2=0x7f0b0c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 --2- 172.21.15.99:0/3611328858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c104e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009900 tx=0x7f0af802bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 -- 172.21.15.99:0/3611328858 shutdown_connections 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 --2- 172.21.15.99:0/3611328858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c105370 0x7f0b0c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 --2- 172.21.15.99:0/3611328858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c104e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 --2- 172.21.15.99:0/3611328858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.359+0000 7f0b11c00640 1 -- 172.21.15.99:0/3611328858 >> 172.21.15.99:0/3611328858 conn(0x7f0b0c0fed50 msgr2=0x7f0b0c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:31.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.360+0000 7f0b11c00640 1 -- 172.21.15.99:0/3611328858 shutdown_connections 2024-03-23T23:43:31.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.360+0000 7f0b11c00640 1 -- 172.21.15.99:0/3611328858 wait complete. 2024-03-23T23:43:31.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.360+0000 7f0b11c00640 1 Processor -- start 2024-03-23T23:43:31.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 -- start start 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 0x7f0b0c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c10f470 con 0x7f0b0c1037c0 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c10f2f0 con 0x7f0b0c105370 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c10f5f0 con 0x7f0b0c1049b0 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b11200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 0x7f0b0c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b0b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39214/0 (socket says 172.21.15.99:39214) 2024-03-23T23:43:31.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.361+0000 7f0b0be00640 1 -- 172.21.15.99:0/74078376 learned_addr learned my addr 172.21.15.99:0/74078376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 -- 172.21.15.99:0/74078376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1049b0 msgr2=0x7f0b0c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 -- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 msgr2=0x7f0b0c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 -- 172.21.15.99:0/74078376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c1a16b0 con 0x7f0b0c105370 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11200640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 0x7f0b0c1a1040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009fd0 tx=0x7f0b0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400ce20 con 0x7f0b0c105370 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b04005020 con 0x7f0b0c105370 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1a19a0 con 0x7f0b0c105370 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04010720 con 0x7f0b0c105370 2024-03-23T23:43:31.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.362+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1a1ec0 con 0x7f0b0c105370 2024-03-23T23:43:31.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.364+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c068800 con 0x7f0b0c105370 2024-03-23T23:43:31.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.366+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0b04005540 con 0x7f0b0c105370 2024-03-23T23:43:31.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.367+0000 7f0b08c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 0x7f0af00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.367+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b040cf170 con 0x7f0b0c105370 2024-03-23T23:43:31.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.367+0000 7f0b0be00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 0x7f0af00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.368+0000 7f0b0be00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 0x7f0af00a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0058c0 tx=0x7f0afc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:31.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.371+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b0409c190 con 0x7f0b0c105370 2024-03-23T23:43:31.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:30 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1659899933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:30 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2877940778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:30 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1659899933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:30 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2877940778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:31.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.596+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b0c0622a0 con 0x7f0b0c105370 2024-03-23T23:43:31.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.597+0000 7f0b08c00640 1 -- 172.21.15.99:0/74078376 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b0409ba90 con 0x7f0b0c105370 2024-03-23T23:43:31.597 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 msgr2=0x7f0af00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 0x7f0af00a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0058c0 tx=0x7f0afc005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 msgr2=0x7f0b0c1a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 0x7f0b0c1a1040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009fd0 tx=0x7f0b0400ef10 comp rx=0 tx=0).stop 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 shutdown_connections 2024-03-23T23:43:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0af00a2880 0x7f0af00a4d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b0c105370 0x7f0b0c1a1040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b0c1049b0 0x7f0b0c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 --2- 172.21.15.99:0/74078376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0c1037c0 0x7f0b0c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 >> 172.21.15.99:0/74078376 conn(0x7f0b0c0fed50 msgr2=0x7f0b0c107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 shutdown_connections 2024-03-23T23:43:31.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.600+0000 7f0b11c00640 1 -- 172.21.15.99:0/74078376 wait complete. 2024-03-23T23:43:31.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 -- 172.21.15.99:0/1248259905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc1073c0 msgr2=0x7fa4bc109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/1248259905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc109810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009ca0 tx=0x7fa4b402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 -- 172.21.15.99:0/1248259905 shutdown_connections 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/1248259905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc109810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/1248259905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc104a90 0x7fa4bc106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/1248259905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc102160 0x7fa4bc104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.828+0000 7fa4c1e00640 1 -- 172.21.15.99:0/1248259905 >> 172.21.15.99:0/1248259905 conn(0x7fa4bc0fbb50 msgr2=0x7fa4bc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.829+0000 7fa4c1e00640 1 -- 172.21.15.99:0/1248259905 shutdown_connections 2024-03-23T23:43:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.829+0000 7fa4c1e00640 1 -- 172.21.15.99:0/1248259905 wait complete. 2024-03-23T23:43:31.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 Processor -- start 2024-03-23T23:43:31.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 -- start start 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc104a90 0x7fa4bc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc10d360 con 0x7fa4bc1073c0 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc10d1e0 con 0x7fa4bc104a90 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4c1e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc10d4e0 con 0x7fa4bc102160 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.830+0000 7fa4baa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41842/0 (socket says 172.21.15.99:41842) 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 -- 172.21.15.99:0/883537726 learned_addr learned my addr 172.21.15.99:0/883537726 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4bb400640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4ba000640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc104a90 0x7fa4bc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 -- 172.21.15.99:0/883537726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc104a90 msgr2=0x7fa4bc19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc104a90 0x7fa4bc19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 -- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc1073c0 msgr2=0x7fa4bc1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:31.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:31.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 -- 172.21.15.99:0/883537726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc1a1810 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4baa00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00e9f0 tx=0x7fa4ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00cd80 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac010910 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.831+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc1a1b00 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.832+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac018990 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.832+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc1a20a0 con 0x7fa4bc102160 2024-03-23T23:43:31.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.833+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc108eb0 con 0x7fa4bc102160 2024-03-23T23:43:31.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.836+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa4ac010ac0 con 0x7fa4bc102160 2024-03-23T23:43:31.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.838+0000 7fa4c0a00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 0x7fa4a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:31.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.839+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa4ac0ced90 con 0x7fa4bc102160 2024-03-23T23:43:31.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.839+0000 7fa4ba000640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 0x7fa4a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:31.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.839+0000 7fa4ba000640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 0x7fa4a00a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80095f0 tx=0x7fa4a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:31.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:31.841+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4ac09b760 con 0x7fa4bc102160 2024-03-23T23:43:31.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:31 smithi099 ceph-mon[31715]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:31.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:31 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1401213839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:31.950 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:31 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/74078376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.100+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4bc02d470 con 0x7fa4bc102160 2024-03-23T23:43:32.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.101+0000 7fa4c0a00640 1 -- 172.21.15.99:0/883537726 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fa4ac0bb020 con 0x7fa4bc102160 2024-03-23T23:43:32.101 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:32.102 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:41:17.278370+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24547,24526,24557,24569,24565]},"id":1}]} 2024-03-23T23:43:32.102 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 26 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 msgr2=0x7fa4a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 0x7fa4a00a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80095f0 tx=0x7fa4a8005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 msgr2=0x7fa4bc199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00e9f0 tx=0x7fa4ac00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 shutdown_connections 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa4a00a2880 0x7fa4a00a4d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4bc1073c0 0x7fa4bc1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4bc104a90 0x7fa4bc19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 --2- 172.21.15.99:0/883537726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102160 0x7fa4bc199bf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 >> 172.21.15.99:0/883537726 conn(0x7fa4bc0fbb50 msgr2=0x7fa4bc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 shutdown_connections 2024-03-23T23:43:32.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.104+0000 7fa4c1e00640 1 -- 172.21.15.99:0/883537726 wait complete. 2024-03-23T23:43:32.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 -- 172.21.15.99:0/3519668966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c1028c0 msgr2=0x7faf6c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 --2- 172.21.15.99:0/3519668966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c1028c0 0x7faf6c104d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf58009900 tx=0x7faf5802bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 -- 172.21.15.99:0/3519668966 shutdown_connections 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 --2- 172.21.15.99:0/3519668966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c105270 0x7faf6c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 --2- 172.21.15.99:0/3519668966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c1028c0 0x7faf6c104d30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 --2- 172.21.15.99:0/3519668966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c0fff50 0x7faf6c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.340+0000 7faf71400640 1 -- 172.21.15.99:0/3519668966 >> 172.21.15.99:0/3519668966 conn(0x7faf6c0fbb50 msgr2=0x7faf6c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.341+0000 7faf71400640 1 -- 172.21.15.99:0/3519668966 shutdown_connections 2024-03-23T23:43:32.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.341+0000 7faf71400640 1 -- 172.21.15.99:0/3519668966 wait complete. 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 Processor -- start 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 -- start start 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 0x7faf6c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c105270 0x7faf6c1a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c1139a0 con 0x7faf6c1028c0 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c113820 con 0x7faf6c0fff50 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.342+0000 7faf71400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c113b20 con 0x7faf6c105270 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34140/0 (socket says 172.21.15.99:34140) 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 -- 172.21.15.99:0/3785221055 learned_addr learned my addr 172.21.15.99:0/3785221055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6b200640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c105270 0x7faf6c1a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf69e00640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 0x7faf6c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 -- 172.21.15.99:0/3785221055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c105270 msgr2=0x7faf6c1a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c105270 0x7faf6c1a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 -- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 msgr2=0x7faf6c19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 0x7faf6c19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 -- 172.21.15.99:0/3785221055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c1a5c00 con 0x7faf6c0fff50 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf69e00640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 0x7faf6c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.343+0000 7faf6a800640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9f0 tx=0x7faf5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.344+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00cd80 con 0x7faf6c0fff50 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.344+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf5c005500 con 0x7faf6c0fff50 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.344+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c010690 con 0x7faf6c0fff50 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.344+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c1a5ef0 con 0x7faf6c0fff50 2024-03-23T23:43:32.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.344+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c1a6300 con 0x7faf6c0fff50 2024-03-23T23:43:32.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.345+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c0fd640 con 0x7faf6c0fff50 2024-03-23T23:43:32.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.348+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faf5c0046c0 con 0x7faf6c0fff50 2024-03-23T23:43:32.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.350+0000 7faf63400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 0x7faf500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.351+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faf5c0ceef0 con 0x7faf6c0fff50 2024-03-23T23:43:32.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.351+0000 7faf69e00640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 0x7faf500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.352+0000 7faf69e00640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 0x7faf500a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf5802c240 tx=0x7faf5802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.353+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf5c09bf10 con 0x7faf6c0fff50 2024-03-23T23:43:32.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:31 smithi149 ceph-mon[36083]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:31 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1401213839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:31 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/74078376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:32.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:31 smithi174 ceph-mon[37002]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:32.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:31 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1401213839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:32.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:31 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/74078376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:32.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.555+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf6c062420 con 0x7faf6c0fff50 2024-03-23T23:43:32.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.557+0000 7faf63400640 1 -- 172.21.15.99:0/3785221055 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7faf5c09b810 con 0x7faf6c0fff50 2024-03-23T23:43:32.560 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:32.561 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 msgr2=0x7faf500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 0x7faf500a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf5802c240 tx=0x7faf5802df90 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 msgr2=0x7faf6c19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9f0 tx=0x7faf5c00edf0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 shutdown_connections 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faf500a2880 0x7faf500a4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf6c105270 0x7faf6c1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf6c1028c0 0x7faf6c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 --2- 172.21.15.99:0/3785221055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf6c0fff50 0x7faf6c19dfe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.559+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 >> 172.21.15.99:0/3785221055 conn(0x7faf6c0fbb50 msgr2=0x7faf6c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.560+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 shutdown_connections 2024-03-23T23:43:32.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.560+0000 7faf71400640 1 -- 172.21.15.99:0/3785221055 wait complete. 2024-03-23T23:43:32.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T23:41:09.728597+0000', 'modified': '2024-03-23T23:41:17.278370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24565, 'mds_2': 24557, 'mds_3': 24547, 'mds_4': 24526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/2919310963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2919310963}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2919310963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/4085463188', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 4085463188}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 4085463188}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1177424904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1177424904}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1177424904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/731827456', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 731827456}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 731827456}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/3655333447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 3655333447}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 3655333447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.174:6837/2363957261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2363957261}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2363957261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.174:6839/1404645838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 1404645838}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 1404645838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.99:6841/678226276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6840', 'nonce': 678226276}, {'type': 'v1', 'addr': '172.21.15.99:6841', 'nonce': 678226276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24565': {'gid': 24565, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/4063961606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 4063961606}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 4063961606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/2464794793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2464794793}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2464794793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24547, 24526, 24557, 24569, 24565]} 2024-03-23T23:43:32.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-23T23:43:32.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-23T23:43:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 -- 172.21.15.99:0/630583986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 msgr2=0x7f2be0104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 --2- 172.21.15.99:0/630583986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0104520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009900 tx=0x7f2bd002bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 -- 172.21.15.99:0/630583986 shutdown_connections 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 --2- 172.21.15.99:0/630583986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0107390 0x7f2be01097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 --2- 172.21.15.99:0/630583986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0104a60 0x7f2be0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 --2- 172.21.15.99:0/630583986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0104520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 -- 172.21.15.99:0/630583986 >> 172.21.15.99:0/630583986 conn(0x7f2be00fbb40 msgr2=0x7f2be00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.768+0000 7f2be4e00640 1 -- 172.21.15.99:0/630583986 shutdown_connections 2024-03-23T23:43:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.769+0000 7f2be4e00640 1 -- 172.21.15.99:0/630583986 wait complete. 2024-03-23T23:43:32.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.769+0000 7f2be4e00640 1 Processor -- start 2024-03-23T23:43:32.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 -- start start 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104a60 0x7f2be01000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0107390 0x7f2be0100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be010d350 con 0x7f2be0104a60 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be010d1d0 con 0x7f2be0102130 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2be4e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be010d4d0 con 0x7f2be0107390 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2bdde00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2bdde00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34158/0 (socket says 172.21.15.99:34158) 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2bdde00640 1 -- 172.21.15.99:0/240755300 learned_addr learned my addr 172.21.15.99:0/240755300 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2bdd400640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104a60 0x7f2be01000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.770+0000 7f2bde800640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0107390 0x7f2be0100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 -- 172.21.15.99:0/240755300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0107390 msgr2=0x7f2be0100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0107390 0x7f2be0100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 -- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104a60 msgr2=0x7f2be01000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104a60 0x7f2be01000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 -- 172.21.15.99:0/240755300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0100eb0 con 0x7f2be0102130 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bdde00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009230 tx=0x7f2bd0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bd6a00640 1 -- 172.21.15.99:0/240755300 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0041070 con 0x7f2be0102130 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bd6a00640 1 -- 172.21.15.99:0/240755300 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd00044d0 con 0x7f2be0102130 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be01a1b50 con 0x7f2be0102130 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2bd6a00640 1 -- 172.21.15.99:0/240755300 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0034440 con 0x7f2be0102130 2024-03-23T23:43:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.771+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01a20a0 con 0x7f2be0102130 2024-03-23T23:43:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.774+0000 7f2be7c1f740 1 Processor -- start 2024-03-23T23:43:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.774+0000 7f2be7c1f740 1 -- start start 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f549090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f54bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bd6a00640 1 -- 172.21.15.99:0/240755300 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2bd0044070 con 0x7f2be0102130 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f4a8c90 con 0x55b01f549700 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f451980 con 0x55b01f544590 2024-03-23T23:43:32.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f4a4d30 con 0x55b01f543ba0 2024-03-23T23:43:32.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41898/0 (socket says 172.21.15.99:41898) 2024-03-23T23:43:32.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 -- 172.21.15.99:0/2011734611 learned_addr learned my addr 172.21.15.99:0/2011734611 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bdde00640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f549090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bdd400640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f54bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 -- 172.21.15.99:0/2011734611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 msgr2=0x55b01f549090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f549090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 -- 172.21.15.99:0/2011734611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 msgr2=0x55b01f54bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f54bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bde800640 1 -- 172.21.15.99:0/2011734611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b01f469510 con 0x55b01f543ba0 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bdde00640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f549090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.775+0000 7f2bdd400640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f54bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.776+0000 7f2bde800640 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80039f0 tx=0x7f2bd8003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7411b6619d745bfc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.776+0000 7f2bcb400640 1 -- 172.21.15.99:0/2011734611 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801a470 con 0x55b01f543ba0 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.776+0000 7f2bcb400640 1 -- 172.21.15.99:0/2011734611 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8015070 con 0x55b01f543ba0 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.776+0000 7f2bcb400640 1 -- 172.21.15.99:0/2011734611 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8010900 con 0x55b01f543ba0 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.776+0000 7f2bd6a00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 0x7f2bc40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2bd6a00640 1 -- 172.21.15.99:0/240755300 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2bd00f06c0 con 0x7f2be0102130 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2bdde00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 0x7f2bc40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2011734611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 msgr2=0x55b01f543fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80039f0 tx=0x7f2bd8003eb0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2011734611 shutdown_connections 2024-03-23T23:43:32.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f54bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2bdde00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 0x7f2bc40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0006010 tx=0x7f2bd00dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f549090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2011734611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f543fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.777+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2011734611 >> 172.21.15.99:0/2011734611 conn(0x55b01f53f110 msgr2=0x55b01f53f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.778+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2011734611 shutdown_connections 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.778+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2011734611 wait complete. 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.778+0000 7f2be7c1f740 1 -- start start 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.778+0000 7f2be7c1f740 1 Processor -- start 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f54e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f5556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f54c6b0 con 0x55b01f549700 2024-03-23T23:43:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f464dd0 con 0x55b01f544590 2024-03-23T23:43:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f54c830 con 0x55b01f543ba0 2024-03-23T23:43:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2be7c1f740 10 client.0 _ll_register_callbacks cb 0x55b01f1bcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2bde800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f54e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34186/0 (socket says 172.21.15.99:34186) 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.779+0000 7f2bdd400640 1 -- 172.21.15.99:0/2919507497 learned_addr learned my addr 172.21.15.99:0/2919507497 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bde800640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f5556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 -- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 msgr2=0x55b01f54e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f54e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 -- 172.21.15.99:0/2919507497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 msgr2=0x55b01f5556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f5556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 -- 172.21.15.99:0/2919507497 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b01f1bd2d0 con 0x55b01f544590 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bde800640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f5556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bdd400640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc012700 tx=0x7f2bcc012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f6d809a1e8fbfa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc004030 con 0x55b01f544590 2024-03-23T23:43:32.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2bcc0041d0 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.780+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd8015070 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bcc0043d0 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 10 client.25166 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b01f2876b0 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8015070 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2bcc003bf0 con 0x55b01f544590 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 10 client.25166 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 10 client.25166 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 20 client.25166 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f55a8e0 0x55b01f5599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 4 client.25166 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b01f44a520 con 0x55b01f55a8e0 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.781+0000 7f2bde800640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f55a8e0 0x55b01f5599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.782+0000 7f2bde800640 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f55a8e0 0x55b01f5599e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52e9bd3b93678158 server_cookie=5b9fbee1badc37e9 in_seq=0 out_seq=0 2024-03-23T23:43:32.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.782+0000 7f2bbfe00640 10 client.25166 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2bbfe00640 1 -- 172.21.15.99:0/2919507497 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b01f44a520 con 0x55b01f55a8e0 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2bbfe00640 10 client.25166 handle_command_reply: tid=0 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 client.25166 shutdown 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 msgr2=0x55b01f54e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc012700 tx=0x7f2bcc012bc0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 shutdown_connections 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f55a8e0 0x55b01f5599e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f549700 0x55b01f5556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f544590 0x55b01f54e660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/2919507497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f543ba0 0x55b01f54e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2bbfe00640 0 client.25166 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 >> 172.21.15.99:0/2919507497 conn(0x55b01f53f110 msgr2=0x55b01f53e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 shutdown_connections 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 1 -- 172.21.15.99:0/2919507497 wait complete. 2024-03-23T23:43:32.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.783+0000 7f2be7c1f740 20 client.25166 trim_cache size 0 max 16384 2024-03-23T23:43:32.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.801+0000 7f2be7c1f740 1 Processor -- start 2024-03-23T23:43:32.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.801+0000 7f2be7c1f740 1 -- start start 2024-03-23T23:43:32.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 0x55b01f5490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f549700 0x55b01f53f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55a100 con 0x55b01f544590 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55a760 con 0x55b01f543ba0 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55adc0 con 0x55b01f549700 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bde800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 0x55b01f5490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39286/0 (socket says 172.21.15.99:39286) 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bdd400640 1 -- 172.21.15.99:0/736641648 learned_addr learned my addr 172.21.15.99:0/736641648 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bde800640 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f549700 0x55b01f53f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.802+0000 7f2bdd400640 1 -- 172.21.15.99:0/736641648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f549700 msgr2=0x55b01f53f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bdd400640 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f549700 0x55b01f53f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bdd400640 1 -- 172.21.15.99:0/736641648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 msgr2=0x55b01f5490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bdd400640 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 0x55b01f5490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bdd400640 1 -- 172.21.15.99:0/736641648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b01f53fa30 con 0x55b01f544590 2024-03-23T23:43:32.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bde800640 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 0x55b01f5490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:32.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bdd400640 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0126d0 tx=0x7f2bcc009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28e3f9ee7b08be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bcaa00640 1 -- 172.21.15.99:0/736641648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0040c0 con 0x55b01f544590 2024-03-23T23:43:32.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bcaa00640 1 -- 172.21.15.99:0/736641648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc00fe00 con 0x55b01f544590 2024-03-23T23:43:32.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.803+0000 7f2bcaa00640 1 -- 172.21.15.99:0/736641648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc003850 con 0x55b01f544590 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- 172.21.15.99:0/736641648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 msgr2=0x55b01f555660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0126d0 tx=0x7f2bcc009fb0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- 172.21.15.99:0/736641648 shutdown_connections 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f549700 0x55b01f53f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f544590 0x55b01f555660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/736641648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f543ba0 0x55b01f5490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- 172.21.15.99:0/736641648 >> 172.21.15.99:0/736641648 conn(0x55b01f53f110 msgr2=0x55b01f54cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- 172.21.15.99:0/736641648 shutdown_connections 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- 172.21.15.99:0/736641648 wait complete. 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.804+0000 7f2be7c1f740 1 -- start start 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 Processor -- start 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 0x55b01f551920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f549700 0x55b01f554e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55ba80 con 0x55b01f543ba0 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55b420 con 0x55b01f549700 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01f55c0e0 con 0x55b01f544590 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2be7c1f740 10 client.0 _ll_register_callbacks cb 0x7f2bcc009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bde800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 0x55b01f551920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bdd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41930/0 (socket says 172.21.15.99:41930) 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bde800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 0x55b01f551920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39300/0 (socket says 172.21.15.99:39300) 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bdd400640 1 -- 172.21.15.99:0/1807608945 learned_addr learned my addr 172.21.15.99:0/1807608945 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.805+0000 7f2bde800640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f549700 0x55b01f554e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 -- 172.21.15.99:0/1807608945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f549700 msgr2=0x55b01f554e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f549700 0x55b01f554e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 -- 172.21.15.99:0/1807608945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 msgr2=0x55b01f551920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 0x55b01f551920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 -- 172.21.15.99:0/1807608945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b01f555470 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bdd400640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005b90 tx=0x7f2bcc009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd60c408624c671d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00fb30 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2bcc0035f0 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bcc00fe00 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bcc0037f0 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2be7c1f740 10 client.25201 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.806+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b01f5b6420 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bcc00fe00 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2bcc010a70 con 0x55b01f544590 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 10 client.25201 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 10 client.25201 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-23T23:43:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 20 client.25201 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f5ba710 0x55b01f5b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 4 client.25201 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b01f4b61d0 con 0x55b01f5ba710 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.807+0000 7f2bde800640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f5ba710 0x55b01f5b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2bbfe00640 10 client.25201 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:43:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2bde800640 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f5ba710 0x55b01f5b6650 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffd13e7b85795c63 server_cookie=78e14888b93ed6d7 in_seq=0 out_seq=0 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2bbfe00640 1 -- 172.21.15.99:0/1807608945 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b01f4b61d0 con 0x55b01f5ba710 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2bbfe00640 10 client.25201 handle_command_reply: tid=0 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 client.25201 shutdown 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 msgr2=0x55b01f551e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005b90 tx=0x7f2bcc009330 comp rx=0 tx=0).stop 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 shutdown_connections 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b01f549700 0x55b01f554e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b01f544590 0x55b01f551e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b01f5ba710 0x55b01f5b6650 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 --2- 172.21.15.99:0/1807608945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b01f543ba0 0x55b01f551920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 >> 172.21.15.99:0/1807608945 conn(0x55b01f53f110 msgr2=0x55b01f539e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.808+0000 7f2bbfe00640 0 client.25201 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 shutdown_connections 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be7c1f740 1 -- 172.21.15.99:0/1807608945 wait complete. 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be7c1f740 20 client.25201 trim_cache size 0 max 16384 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 4, 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24526, 2024-03-23T23:43:32.813 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 136.07394159699999, 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 166.15295662099999 2024-03-23T23:43:32.814 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 msgr2=0x7f2bc40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 0x7f2bc40a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0006010 tx=0x7f2bd00dd040 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 msgr2=0x7f2be0101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.809+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009230 tx=0x7f2bd0005d50 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 shutdown_connections 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2bc40a2800 0x7f2bc40a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be0107390 0x7f2be0100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0104a60 0x7f2be01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 --2- 172.21.15.99:0/240755300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0102130 0x7f2be0101aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:32.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 >> 172.21.15.99:0/240755300 conn(0x7f2be00fbb40 msgr2=0x7f2be0102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:32.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 shutdown_connections 2024-03-23T23:43:32.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:32.810+0000 7f2be4e00640 1 -- 172.21.15.99:0/240755300 wait complete. 2024-03-23T23:43:32.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 -- 172.21.15.99:0/1754607954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20500772a0 msgr2=0x7f2050077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 --2- 172.21.15.99:0/1754607954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20500772a0 0x7f2050077700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2038009900 tx=0x7f203802bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 -- 172.21.15.99:0/1754607954 shutdown_connections 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 --2- 172.21.15.99:0/1754607954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050077c40 0x7f20500784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 --2- 172.21.15.99:0/1754607954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20500772a0 0x7f2050077700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 --2- 172.21.15.99:0/1754607954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050076060 0x7f2050076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 -- 172.21.15.99:0/1754607954 >> 172.21.15.99:0/1754607954 conn(0x7f20500fed10 msgr2=0x7f2050101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 -- 172.21.15.99:0/1754607954 shutdown_connections 2024-03-23T23:43:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.022+0000 7f204fe00640 1 -- 172.21.15.99:0/1754607954 wait complete. 2024-03-23T23:43:33.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.023+0000 7f204fe00640 1 Processor -- start 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.023+0000 7f204fe00640 1 -- start start 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 0x7f20501a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050113870 con 0x7f20500772a0 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501136f0 con 0x7f2050077c40 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501139f0 con 0x7f2050076060 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f2047e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f2047e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39320/0 (socket says 172.21.15.99:39320) 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f2047e00640 1 -- 172.21.15.99:0/159038790 learned_addr learned my addr 172.21.15.99:0/159038790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204f400640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 0x7f20501a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.024+0000 7f204ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41942/0 (socket says 172.21.15.99:41942) 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 -- 172.21.15.99:0/159038790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 msgr2=0x7f20501a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 0x7f20501a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 -- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 msgr2=0x7f205019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 -- 172.21.15.99:0/159038790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20501a5ab0 con 0x7f2050076060 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204f400640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 0x7f20501a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f2047e00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204ea00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f204000e9f0 tx=0x7f204000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f2047400640 1 -- 172.21.15.99:0/159038790 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204000ccc0 con 0x7f2050076060 2024-03-23T23:43:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f2047400640 1 -- 172.21.15.99:0/159038790 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2040005020 con 0x7f2050076060 2024-03-23T23:43:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501a5da0 con 0x7f2050076060 2024-03-23T23:43:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f2047400640 1 -- 172.21.15.99:0/159038790 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400049e0 con 0x7f2050076060 2024-03-23T23:43:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.025+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a6200 con 0x7f2050076060 2024-03-23T23:43:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.028+0000 7f2047400640 1 -- 172.21.15.99:0/159038790 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2040005560 con 0x7f2050076060 2024-03-23T23:43:33.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.029+0000 7f205820e740 1 Processor -- start 2024-03-23T23:43:33.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.029+0000 7f205820e740 1 -- start start 2024-03-23T23:43:33.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562454730c90 con 0x5624547cc770 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624546d9980 con 0x5624547d18e0 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f205820e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245472cd30 con 0x5624547cbd80 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41956/0 (socket says 172.21.15.99:41956) 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 -- 172.21.15.99:0/3294435560 learned_addr learned my addr 172.21.15.99:0/3294435560 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f2047e00640 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 -- 172.21.15.99:0/3294435560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 msgr2=0x5624547d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 -- 172.21.15.99:0/3294435560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 msgr2=0x5624547d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f204f400640 1 -- 172.21.15.99:0/3294435560 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624546f1510 con 0x5624547cbd80 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.030+0000 7f2047e00640 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f2047400640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 0x7f20340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f204f400640 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20480117e0 tx=0x7f2048011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3649c2af6ff6a90 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f204ea00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 0x7f20340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f2047400640 1 -- 172.21.15.99:0/159038790 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f20400ce9d0 con 0x7f2050076060 2024-03-23T23:43:33.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f203f400640 1 -- 172.21.15.99:0/3294435560 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048011310 con 0x5624547cbd80 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f203f400640 1 -- 172.21.15.99:0/3294435560 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2048003e50 con 0x5624547cbd80 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.031+0000 7f203f400640 1 -- 172.21.15.99:0/3294435560 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048004620 con 0x5624547cbd80 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- 172.21.15.99:0/3294435560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 msgr2=0x5624547cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20480117e0 tx=0x7f2048011ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f204ea00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 0x7f20340a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20400a3560 tx=0x7f20400d4530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- 172.21.15.99:0/3294435560 shutdown_connections 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 --2- 172.21.15.99:0/3294435560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547cc1a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- 172.21.15.99:0/3294435560 >> 172.21.15.99:0/3294435560 conn(0x5624547c72f0 msgr2=0x5624547c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- 172.21.15.99:0/3294435560 shutdown_connections 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- 172.21.15.99:0/3294435560 wait complete. 2024-03-23T23:43:33.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.032+0000 7f205820e740 1 -- start start 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.033+0000 7f205820e740 1 Processor -- start 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.033+0000 7f205820e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.033+0000 7f205820e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.033+0000 7f205820e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547dd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.033+0000 7f205820e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547d4890 con 0x5624547cc770 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f205820e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624546ecdd0 con 0x5624547d18e0 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f205820e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547d4a10 con 0x5624547cbd80 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f205820e740 10 client.0 _ll_register_callbacks cb 0x562454444ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f204f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f204f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41966/0 (socket says 172.21.15.99:41966) 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f204f400640 1 -- 172.21.15.99:0/3737493023 learned_addr learned my addr 172.21.15.99:0/3737493023 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f204f400640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547dd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 -- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 msgr2=0x5624547d62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 -- 172.21.15.99:0/3737493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 msgr2=0x5624547dd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547dd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f2047e00640 1 -- 172.21.15.99:0/3737493023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624544452d0 con 0x5624547cc770 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.034+0000 7f204f400640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.035+0000 7f2047e00640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d6800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2038005ab0 tx=0x7f2038004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c023ac29cb7d849 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.035+0000 7f2033e00640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:43:33.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.035+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038005d50 con 0x5624547cc770 2024-03-23T23:43:33.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.035+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2038004ed0 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.035+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2048003e50 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2038050070 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f205820e740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56245450f6b0 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2048003e50 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2038003800 con 0x5624547cc770 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f205820e740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f205820e740 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.036+0000 7f205820e740 20 client.15087 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5624547e2a40 0x5624547e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f205820e740 4 client.15087 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624546d2520 con 0x5624547e2a40 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f204f400640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5624547e2a40 0x5624547e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f204f400640 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5624547e2a40 0x5624547e1b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8100aaeea1abc44f server_cookie=e9af08a81e0b82b in_seq=0 out_seq=0 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.037+0000 7f2033e00640 10 client.15087 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.038+0000 7f2033e00640 1 -- 172.21.15.99:0/3737493023 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5624546d2520 con 0x5624547e2a40 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.038+0000 7f2033e00640 10 client.15087 handle_command_reply: tid=0 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.038+0000 7f205820e740 1 client.15087 shutdown 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 msgr2=0x5624547d6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d6800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2038005ab0 tx=0x7f2038004970 comp rx=0 tx=0).stop 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 shutdown_connections 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5624547e2a40 0x5624547e1b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547d18e0 0x5624547dd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547d6800 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 --2- 172.21.15.99:0/3737493023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cbd80 0x5624547d62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 >> 172.21.15.99:0/3737493023 conn(0x5624547c72f0 msgr2=0x5624547c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f2033e00640 0 client.15087 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 shutdown_connections 2024-03-23T23:43:33.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 1 -- 172.21.15.99:0/3737493023 wait complete. 2024-03-23T23:43:33.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.039+0000 7f205820e740 20 client.15087 trim_cache size 0 max 16384 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 Processor -- start 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 -- start start 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547d18e0 0x5624547d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e2260 con 0x5624547cc770 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e28c0 con 0x5624547cbd80 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.074+0000 7f205820e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e2f20 con 0x5624547d18e0 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f204f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39354/0 (socket says 172.21.15.99:39354) 2024-03-23T23:43:33.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 -- 172.21.15.99:0/3294122221 learned_addr learned my addr 172.21.15.99:0/3294122221 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f204f400640 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547d18e0 0x5624547d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 -- 172.21.15.99:0/3294122221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547d18e0 msgr2=0x5624547d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547d18e0 0x5624547d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 -- 172.21.15.99:0/3294122221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 msgr2=0x5624547d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 -- 172.21.15.99:0/3294122221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624547d0f60 con 0x5624547cc770 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f204f400640 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.075+0000 7f2047e00640 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2038006010 tx=0x7f203802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3708cd62e9a5c35e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f203ea00640 1 -- 172.21.15.99:0/3294122221 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038005d50 con 0x5624547cc770 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f203ea00640 1 -- 172.21.15.99:0/3294122221 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203802cd30 con 0x5624547cc770 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f203ea00640 1 -- 172.21.15.99:0/3294122221 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203804a480 con 0x5624547cc770 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 -- 172.21.15.99:0/3294122221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 msgr2=0x5624547dd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2038006010 tx=0x7f203802bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 -- 172.21.15.99:0/3294122221 shutdown_connections 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547d18e0 0x5624547d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547cc770 0x5624547dd800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 --2- 172.21.15.99:0/3294122221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.076+0000 7f205820e740 1 -- 172.21.15.99:0/3294122221 >> 172.21.15.99:0/3294122221 conn(0x5624547c72f0 msgr2=0x5624547cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.077+0000 7f205820e740 1 -- 172.21.15.99:0/3294122221 shutdown_connections 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.077+0000 7f205820e740 1 -- 172.21.15.99:0/3294122221 wait complete. 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.077+0000 7f205820e740 1 -- start start 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.077+0000 7f205820e740 1 Processor -- start 2024-03-23T23:43:33.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547d18e0 0x5624547dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e3be0 con 0x5624547d18e0 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e3580 con 0x5624547cbd80 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624547e4240 con 0x5624547cc770 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f205820e740 10 client.0 _ll_register_callbacks cb 0x7f2040005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f204f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41982/0 (socket says 172.21.15.99:41982) 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 -- 172.21.15.99:0/3896487999 learned_addr learned my addr 172.21.15.99:0/3896487999 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f204f400640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547d18e0 0x5624547dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 -- 172.21.15.99:0/3896487999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 msgr2=0x5624547d8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 -- 172.21.15.99:0/3896487999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547d18e0 msgr2=0x5624547dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547d18e0 0x5624547dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 -- 172.21.15.99:0/3896487999 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624547dcb40 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2047e00640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f203802c8e0 tx=0x7f2038005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c4b13be990676a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.078+0000 7f2033e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038061070 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f203802cd30 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2038004ed0 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203806c050 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f205820e740 10 client.25219 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56245483db00 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.079+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203802cd30 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2038055510 con 0x5624547cc770 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 10 client.25219 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 10 client.25219 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T23:43:33.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 20 client.25219 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562454841ca0 0x56245483dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 4 client.25219 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56245473e1d0 con 0x562454841ca0 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.080+0000 7f204f400640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562454841ca0 0x56245483dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f2033e00640 10 client.25219 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f204f400640 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562454841ca0 0x56245483dcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=852b7cb1bf6333f3 server_cookie=9959737399f1e2ba in_seq=0 out_seq=0 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f2033e00640 1 -- 172.21.15.99:0/3896487999 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56245473e1d0 con 0x562454841ca0 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f2033e00640 10 client.25219 handle_command_reply: tid=0 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 client.25219 shutdown 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 msgr2=0x5624547d9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f203802c8e0 tx=0x7f2038005a20 comp rx=0 tx=0).stop 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 shutdown_connections 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x562454841ca0 0x56245483dcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624547d18e0 0x5624547dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f2033e00640 0 client.25219 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624547cc770 0x5624547d9530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 --2- 172.21.15.99:0/3896487999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624547cbd80 0x5624547d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 >> 172.21.15.99:0/3896487999 conn(0x5624547c72f0 msgr2=0x5624547c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 shutdown_connections 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 1 -- 172.21.15.99:0/3896487999 wait complete. 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.081+0000 7f205820e740 20 client.25219 trim_cache size 0 max 16384 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 3, 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24547, 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:43:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 137.36136200000001, 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 160.629055475 2024-03-23T23:43:33.086 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 msgr2=0x7f20340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 0x7f20340a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f20400a3560 tx=0x7f20400d4530 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 msgr2=0x7f205019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f204000e9f0 tx=0x7f204000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 shutdown_connections 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f20340a2880 0x7f20340a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050077c40 0x7f20501a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500772a0 0x7f205019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 --2- 172.21.15.99:0/159038790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2050076060 0x7f205019de90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 >> 172.21.15.99:0/159038790 conn(0x7f20500fed10 msgr2=0x7f2050100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 shutdown_connections 2024-03-23T23:43:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.082+0000 7f204fe00640 1 -- 172.21.15.99:0/159038790 wait complete. 2024-03-23T23:43:33.102 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T23:43:33.170 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:32 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/883537726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.170 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:32 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3785221055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 -- 172.21.15.99:0/1268552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5901015a0 msgr2=0x7fa5901019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 --2- 172.21.15.99:0/1268552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5901015a0 0x7fa5901019c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa584009900 tx=0x7fa58402bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 -- 172.21.15.99:0/1268552558 shutdown_connections 2024-03-23T23:43:33.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 --2- 172.21.15.99:0/1268552558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa590103150 0x7fa5901099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 --2- 172.21.15.99:0/1268552558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 0x7fa590102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 --2- 172.21.15.99:0/1268552558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5901015a0 0x7fa5901019c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 -- 172.21.15.99:0/1268552558 >> 172.21.15.99:0/1268552558 conn(0x7fa5900fcb10 msgr2=0x7fa5900fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 -- 172.21.15.99:0/1268552558 shutdown_connections 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.317+0000 7fa596a00640 1 -- 172.21.15.99:0/1268552558 wait complete. 2024-03-23T23:43:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.319+0000 7fa596a00640 1 Processor -- start 2024-03-23T23:43:33.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.319+0000 7fa596a00640 1 -- start start 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.319+0000 7fa596a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.319+0000 7fa596a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 0x7fa590197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa596a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590103150 0x7fa59019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa596a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59010d260 con 0x7fa590102790 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa596a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59010d0e0 con 0x7fa5901015a0 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa596a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59010d3e0 con 0x7fa590103150 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa594c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 0x7fa590197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34280/0 (socket says 172.21.15.99:34280) 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 -- 172.21.15.99:0/90425403 learned_addr learned my addr 172.21.15.99:0/90425403 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa596000640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590103150 0x7fa59019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 -- 172.21.15.99:0/90425403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590103150 msgr2=0x7fa59019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590103150 0x7fa59019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 -- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 msgr2=0x7fa590197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 0x7fa590197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 -- 172.21.15.99:0/90425403 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59019f4c0 con 0x7fa5901015a0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.320+0000 7fa595600640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa58402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.321+0000 7fa58e000640 1 -- 172.21.15.99:0/90425403 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584043020 con 0x7fa5901015a0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.321+0000 7fa58e000640 1 -- 172.21.15.99:0/90425403 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5840054e0 con 0x7fa5901015a0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.321+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59019f750 con 0x7fa5901015a0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.321+0000 7fa58e000640 1 -- 172.21.15.99:0/90425403 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584003c40 con 0x7fa5901015a0 2024-03-23T23:43:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.321+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59019fca0 con 0x7fa5901015a0 2024-03-23T23:43:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.324+0000 7fa58e000640 1 -- 172.21.15.99:0/90425403 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa584046020 con 0x7fa5901015a0 2024-03-23T23:43:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.324+0000 7fa59ade0740 1 Processor -- start 2024-03-23T23:43:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.324+0000 7fa59ade0740 1 -- start start 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da8091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da80e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da76dc90 con 0x55a7da808d80 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da716980 con 0x55a7da809770 2024-03-23T23:43:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa59ade0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da769d30 con 0x55a7da80e8e0 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa595600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da80e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa596000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da8091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa595600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da80e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34294/0 (socket says 172.21.15.99:34294) 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42020/0 (socket says 172.21.15.99:42020) 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa595600640 1 -- 172.21.15.99:0/2478983718 learned_addr learned my addr 172.21.15.99:0/2478983718 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa596000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da8091a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39394/0 (socket says 172.21.15.99:39394) 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 -- 172.21.15.99:0/2478983718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 msgr2=0x55a7da80e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da80e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 -- 172.21.15.99:0/2478983718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 msgr2=0x55a7da8091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da8091a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa594c00640 1 -- 172.21.15.99:0/2478983718 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7da72e510 con 0x55a7da80e8e0 2024-03-23T23:43:33.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.325+0000 7fa58e000640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 0x7fa5780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa594c00640 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009bf0 tx=0x7fa57c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b0d7cdcf8ccf559 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa58e000640 1 -- 172.21.15.99:0/90425403 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5840f0a20 con 0x7fa5901015a0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa596000640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 0x7fa5780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa582600640 1 -- 172.21.15.99:0/2478983718 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004050 con 0x55a7da80e8e0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa582600640 1 -- 172.21.15.99:0/2478983718 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa57c005500 con 0x55a7da80e8e0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.326+0000 7fa582600640 1 -- 172.21.15.99:0/2478983718 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c004de0 con 0x55a7da80e8e0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- 172.21.15.99:0/2478983718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 msgr2=0x55a7da810d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009bf0 tx=0x7fa57c011810 comp rx=0 tx=0).stop 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa596000640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 0x7fa5780a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a7da809fd0 tx=0x7fa588009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- 172.21.15.99:0/2478983718 shutdown_connections 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da810d70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da80e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2478983718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da8091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- 172.21.15.99:0/2478983718 >> 172.21.15.99:0/2478983718 conn(0x55a7da8042f0 msgr2=0x55a7da8046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- 172.21.15.99:0/2478983718 shutdown_connections 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- 172.21.15.99:0/2478983718 wait complete. 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.327+0000 7fa59ade0740 1 -- start start 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 Processor -- start 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da817650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da811890 con 0x55a7da808d80 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da729dd0 con 0x55a7da809770 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da811a10 con 0x55a7da80e8e0 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa59ade0740 10 client.0 _ll_register_callbacks cb 0x55a7da481ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34306/0 (socket says 172.21.15.99:34306) 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa594c00640 1 -- 172.21.15.99:0/3578554171 learned_addr learned my addr 172.21.15.99:0/3578554171 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa596000640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.328+0000 7fa595600640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da817650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 -- 172.21.15.99:0/3578554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 msgr2=0x55a7da81ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 -- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 msgr2=0x55a7da817650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da817650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 -- 172.21.15.99:0/3578554171 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7da4822d0 con 0x55a7da809770 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa596000640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa595600640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da817650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa594c00640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa57c01b040 tx=0x7fa57c009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90e6bdb1f5fef16 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa576a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c029070 con 0x55a7da809770 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa57c005500 con 0x55a7da809770 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa57c004d60 con 0x55a7da809770 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa57c035050 con 0x55a7da809770 2024-03-23T23:43:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.329+0000 7fa59ade0740 10 client.25202 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7da54c6b0 con 0x55a7da809770 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa57c005500 con 0x55a7da809770 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa57c023430 con 0x55a7da809770 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 10 client.25202 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 20 client.25202 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da823b70 0x55a7da822de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 4 client.25202 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7da70f520 con 0x55a7da823b70 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.330+0000 7fa595600640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da823b70 0x55a7da822de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.331+0000 7fa595600640 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da823b70 0x55a7da822de0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=945da2954d45e372 server_cookie=47b31eb862809afa in_seq=0 out_seq=0 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.331+0000 7fa576a00640 10 client.25202 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.331+0000 7fa576a00640 1 -- 172.21.15.99:0/3578554171 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a7da70f520 con 0x55a7da823b70 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.331+0000 7fa576a00640 10 client.25202 handle_command_reply: tid=0 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 client.25202 shutdown 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 msgr2=0x55a7da817b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa57c01b040 tx=0x7fa57c009850 comp rx=0 tx=0).stop 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 shutdown_connections 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da823b70 0x55a7da822de0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da817b90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 --2- 172.21.15.99:0/3578554171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da817650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 >> 172.21.15.99:0/3578554171 conn(0x55a7da8042f0 msgr2=0x55a7da8034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa576a00640 0 client.25202 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 shutdown_connections 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 1 -- 172.21.15.99:0/3578554171 wait complete. 2024-03-23T23:43:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.332+0000 7fa59ade0740 20 client.25202 trim_cache size 0 max 16384 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 Processor -- start 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 -- start start 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da810cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da817260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da823390 con 0x55a7da808d80 2024-03-23T23:43:33.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da8239f0 con 0x55a7da809770 2024-03-23T23:43:33.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.350+0000 7fa59ade0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da824050 con 0x55a7da80e8e0 2024-03-23T23:43:33.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa595600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da810cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34314/0 (socket says 172.21.15.99:34314) 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 -- 172.21.15.99:0/2751248417 learned_addr learned my addr 172.21.15.99:0/2751248417 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa596000640 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da817260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 -- 172.21.15.99:0/2751248417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 msgr2=0x55a7da817260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da817260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 -- 172.21.15.99:0/2751248417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 msgr2=0x55a7da810cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da810cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 -- 172.21.15.99:0/2751248417 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7da80e620 con 0x55a7da809770 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa594c00640 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0059f0 tx=0x7fa57c009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1feb3ba919875de4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa581c00640 1 -- 172.21.15.99:0/2751248417 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c029070 con 0x55a7da809770 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.351+0000 7fa581c00640 1 -- 172.21.15.99:0/2751248417 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa57c023a40 con 0x55a7da809770 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa581c00640 1 -- 172.21.15.99:0/2751248417 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c02e440 con 0x55a7da809770 2024-03-23T23:43:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 -- 172.21.15.99:0/2751248417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 msgr2=0x55a7da81eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0059f0 tx=0x7fa57c009af0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 -- 172.21.15.99:0/2751248417 shutdown_connections 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da817260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eb90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2751248417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da810cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 -- 172.21.15.99:0/2751248417 >> 172.21.15.99:0/2751248417 conn(0x55a7da8042f0 msgr2=0x55a7da8046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 -- 172.21.15.99:0/2751248417 shutdown_connections 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.352+0000 7fa59ade0740 1 -- 172.21.15.99:0/2751248417 wait complete. 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 -- start start 2024-03-23T23:43:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 Processor -- start 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da81e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da824d10 con 0x55a7da808d80 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da8246b0 con 0x55a7da809770 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7da825370 con 0x55a7da80e8e0 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa59ade0740 10 client.0 _ll_register_callbacks cb 0x7fa588009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa595600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da81e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa596000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa594c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34326/0 (socket says 172.21.15.99:34326) 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.353+0000 7fa594c00640 1 -- 172.21.15.99:0/2168543832 learned_addr learned my addr 172.21.15.99:0/2168543832 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 -- 172.21.15.99:0/2168543832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 msgr2=0x55a7da81b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 -- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 msgr2=0x55a7da81e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da81e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 -- 172.21.15.99:0/2168543832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7da81b820 con 0x55a7da809770 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa594c00640 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0238d0 tx=0x7fa57c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e8b720c8fec4e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa576a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c03d070 con 0x55a7da809770 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa57c0037f0 con 0x55a7da809770 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.354+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa57c023a40 con 0x55a7da809770 2024-03-23T23:43:33.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa57c049050 con 0x55a7da809770 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 10 client.25208 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a7da87f910 con 0x55a7da809770 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa57c023a40 con 0x55a7da809770 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fa57c039430 con 0x55a7da809770 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 10 client.25208 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 10 client.25208 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 20 client.25208 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da883c20 0x55a7da87fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 4 client.25208 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7da77b1d0 con 0x55a7da883c20 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.355+0000 7fa595600640 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da883c20 0x55a7da87fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa595600640 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da883c20 0x55a7da87fb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb2f6c0740009266 server_cookie=5d26de4dd0ff92c in_seq=0 out_seq=0 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa576a00640 10 client.25208 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:43:33.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa576a00640 1 -- 172.21.15.99:0/2168543832 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a7da77b1d0 con 0x55a7da883c20 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa576a00640 10 client.25208 handle_command_reply: tid=0 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 client.25208 shutdown 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 msgr2=0x55a7da81eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0238d0 tx=0x7fa57c005fd0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 shutdown_connections 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55a7da883c20 0x55a7da87fb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.356+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7da80e8e0 0x55a7da81b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa576a00640 0 client.25208 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7da809770 0x55a7da81eac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 1 --2- 172.21.15.99:0/2168543832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7da808d80 0x55a7da81e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 >> 172.21.15.99:0/2168543832 conn(0x55a7da8042f0 msgr2=0x55a7da817260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 shutdown_connections 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 1 -- 172.21.15.99:0/2168543832 wait complete. 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa59ade0740 20 client.25208 trim_cache size 0 max 16384 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24557, 2024-03-23T23:43:33.361 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 138.64650328600001, 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 152.78267555400001 2024-03-23T23:43:33.362 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 msgr2=0x7fa5780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 0x7fa5780a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a7da809fd0 tx=0x7fa588009500 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 msgr2=0x7fa5901978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.357+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa584000c00 tx=0x7fa58402c650 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 shutdown_connections 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa5780a2800 0x7fa5780a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590103150 0x7fa59019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa590102790 0x7fa590197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 --2- 172.21.15.99:0/90425403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5901015a0 0x7fa5901978a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 >> 172.21.15.99:0/90425403 conn(0x7fa5900fcb10 msgr2=0x7fa590105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 shutdown_connections 2024-03-23T23:43:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.358+0000 7fa596a00640 1 -- 172.21.15.99:0/90425403 wait complete. 2024-03-23T23:43:33.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-23T23:43:33.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:32 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/883537726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:32 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3785221055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:32 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/883537726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:32 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3785221055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 -- 172.21.15.99:0/1220384249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e41037a0 msgr2=0x7fb1e4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 --2- 172.21.15.99:0/1220384249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e41037a0 0x7fb1e4103bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009ca0 tx=0x7fb1d402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 -- 172.21.15.99:0/1220384249 shutdown_connections 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 --2- 172.21.15.99:0/1220384249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e4105350 0x7fb1e410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 --2- 172.21.15.99:0/1220384249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 --2- 172.21.15.99:0/1220384249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e41037a0 0x7fb1e4103bc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 -- 172.21.15.99:0/1220384249 >> 172.21.15.99:0/1220384249 conn(0x7fb1e40fed10 msgr2=0x7fb1e4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 -- 172.21.15.99:0/1220384249 shutdown_connections 2024-03-23T23:43:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.576+0000 7fb1e3200640 1 -- 172.21.15.99:0/1220384249 wait complete. 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.577+0000 7fb1e3200640 1 Processor -- start 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.577+0000 7fb1e3200640 1 -- start start 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.577+0000 7fb1e3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.577+0000 7fb1e3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e4105350 0x7fb1e41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e3200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e410f3f0 con 0x7fb1e4104990 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e410f270 con 0x7fb1e4105350 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e410f570 con 0x7fb1e41037a0 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42084/0 (socket says 172.21.15.99:42084) 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 -- 172.21.15.99:0/2863258902 learned_addr learned my addr 172.21.15.99:0/2863258902 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1d9400640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e2800640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e4105350 0x7fb1e41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 -- 172.21.15.99:0/2863258902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e4105350 msgr2=0x7fb1e41a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e4105350 0x7fb1e41a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 -- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 msgr2=0x7fb1e419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 -- 172.21.15.99:0/2863258902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e41a1710 con 0x7fb1e41037a0 2024-03-23T23:43:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1d9400640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.578+0000 7fb1e1e00640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4000f80 tx=0x7fb1d4032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.579+0000 7fb1db400640 1 -- 172.21.15.99:0/2863258902 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4041020 con 0x7fb1e41037a0 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.579+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e41a19a0 con 0x7fb1e41037a0 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.579+0000 7fb1db400640 1 -- 172.21.15.99:0/2863258902 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1d4034cd0 con 0x7fb1e41037a0 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.579+0000 7fb1db400640 1 -- 172.21.15.99:0/2863258902 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4039040 con 0x7fb1e41037a0 2024-03-23T23:43:33.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.579+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e41a1e60 con 0x7fb1e41037a0 2024-03-23T23:43:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 Processor -- start 2024-03-23T23:43:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 -- start start 2024-03-23T23:43:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1db400640 1 -- 172.21.15.99:0/2863258902 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb1d4044020 con 0x7fb1e41037a0 2024-03-23T23:43:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4f770 0x5589fbd54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 0x5589fbd56d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbcb3c90 con 0x5589fbd548e0 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbc5c980 con 0x5589fbd4f770 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.582+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbcafd30 con 0x5589fbd4ed80 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4f770 0x5589fbd54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42096/0 (socket says 172.21.15.99:42096) 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 -- 172.21.15.99:0/1147517322 learned_addr learned my addr 172.21.15.99:0/1147517322 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1d9400640 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 0x5589fbd56d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 -- 172.21.15.99:0/1147517322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4f770 msgr2=0x5589fbd54270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4f770 0x5589fbd54270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 -- 172.21.15.99:0/1147517322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 msgr2=0x5589fbd56d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 0x5589fbd56d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 -- 172.21.15.99:0/1147517322 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589fbc74510 con 0x5589fbd4ed80 2024-03-23T23:43:33.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1d9400640 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 0x5589fbd56d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:33.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.583+0000 7fb1e2800640 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c90 tx=0x7fb1dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95a5bed512be678 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1d3400640 1 -- 172.21.15.99:0/1147517322 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004230 con 0x5589fbd4ed80 2024-03-23T23:43:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1d3400640 1 -- 172.21.15.99:0/1147517322 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1dc009e40 con 0x5589fbd4ed80 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1d3400640 1 -- 172.21.15.99:0/1147517322 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0197e0 con 0x5589fbd4ed80 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1db400640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 0x7fb1c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1db400640 1 -- 172.21.15.99:0/2863258902 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb1d40f0910 con 0x7fb1e41037a0 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/1147517322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 msgr2=0x5589fbd4f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c90 tx=0x7fb1dc011810 comp rx=0 tx=0).stop 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.584+0000 7fb1e1e00640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 0x7fb1c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/1147517322 shutdown_connections 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd548e0 0x5589fbd56d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4f770 0x5589fbd54270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/1147517322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd4f1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/1147517322 >> 172.21.15.99:0/1147517322 conn(0x5589fbd4a2f0 msgr2=0x5589fbd4a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/1147517322 shutdown_connections 2024-03-23T23:43:33.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/1147517322 wait complete. 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1e1e00640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 0x7fb1c80a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40bd250 tx=0x7fb1d40f6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.585+0000 7fb1ebfa3740 1 -- start start 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 Processor -- start 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd57890 con 0x5589fbd4f770 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbc6fdd0 con 0x5589fbd548e0 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd57a10 con 0x5589fbd4ed80 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1ebfa3740 10 client.0 _ll_register_callbacks cb 0x5589fb9c7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42112/0 (socket says 172.21.15.99:42112) 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.586+0000 7fb1e2800640 1 -- 172.21.15.99:0/2749981141 learned_addr learned my addr 172.21.15.99:0/2749981141 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1d9400640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 -- 172.21.15.99:0/2749981141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 msgr2=0x5589fbd60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 -- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 msgr2=0x5589fbd597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 -- 172.21.15.99:0/2749981141 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589fb9c82d0 con 0x5589fbd4ed80 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1e2800640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0038b0 tx=0x7fb1dc009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feacc26e8572d875 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc02d060 con 0x5589fbd4ed80 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb1dc003b20 con 0x5589fbd4ed80 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.587+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589fbc55520 con 0x5589fbd4ed80 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1dc005050 con 0x5589fbd4ed80 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 10 client.25249 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5589fba926b0 con 0x5589fbd4ed80 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5589fbc55520 con 0x5589fbd4ed80 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb1dc018070 con 0x5589fbd4ed80 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 10 client.25249 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 10 client.25249 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 20 client.25249 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbd65780 0x5589fbd69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 4 client.25249 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.588+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589fbcc11d0 con 0x5589fbd65780 2024-03-23T23:43:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.589+0000 7fb1d9400640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbd65780 0x5589fbd69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.589+0000 7fb1d9400640 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbd65780 0x5589fbd69c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71435a6db4bfe893 server_cookie=4492f8d887d40505 in_seq=0 out_seq=0 2024-03-23T23:43:33.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.589+0000 7fb1c7e00640 10 client.25249 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:43:33.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.590+0000 7fb1c7e00640 1 -- 172.21.15.99:0/2749981141 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5589fbcc11d0 con 0x5589fbd65780 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.590+0000 7fb1c7e00640 10 client.25249 handle_command_reply: tid=0 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.590+0000 7fb1ebfa3740 1 client.25249 shutdown 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 msgr2=0x5589fbd592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0038b0 tx=0x7fb1dc009760 comp rx=0 tx=0).stop 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 shutdown_connections 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbd65780 0x5589fbd69c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1c7e00640 0 client.25249 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/2749981141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4ed80 0x5589fbd592b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 >> 172.21.15.99:0/2749981141 conn(0x5589fbd4a2f0 msgr2=0x5589fbd494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 shutdown_connections 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/2749981141 wait complete. 2024-03-23T23:43:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.591+0000 7fb1ebfa3740 20 client.25249 trim_cache size 0 max 16384 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 Processor -- start 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 -- start start 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4ed80 0x5589fbd57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd64fa0 con 0x5589fbd4ed80 2024-03-23T23:43:33.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd65600 con 0x5589fbd548e0 2024-03-23T23:43:33.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.609+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd65c60 con 0x5589fbd4f770 2024-03-23T23:43:33.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42128/0 (socket says 172.21.15.99:42128) 2024-03-23T23:43:33.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 -- 172.21.15.99:0/3884269928 learned_addr learned my addr 172.21.15.99:0/3884269928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1e2800640 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4ed80 0x5589fbd57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1e2800640 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 -- 172.21.15.99:0/3884269928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 msgr2=0x5589fbd4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 -- 172.21.15.99:0/3884269928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4ed80 msgr2=0x5589fbd57db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4ed80 0x5589fbd57db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 -- 172.21.15.99:0/3884269928 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589fbd3c170 con 0x5589fbd4f770 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d9400640 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5589fbd5a180 tx=0x7fb1cc0097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5215973846a0faa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d2a00640 1 -- 172.21.15.99:0/3884269928 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x5589fbd4f770 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d2a00640 1 -- 172.21.15.99:0/3884269928 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc003c70 con 0x5589fbd4f770 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.610+0000 7fb1d2a00640 1 -- 172.21.15.99:0/3884269928 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc012d00 con 0x5589fbd4f770 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/3884269928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 msgr2=0x5589fbd607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5589fbd5a180 tx=0x7fb1cc0097f0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/3884269928 shutdown_connections 2024-03-23T23:43:33.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd548e0 0x5589fbd4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd4f770 0x5589fbd607c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/3884269928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4ed80 0x5589fbd57db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/3884269928 >> 172.21.15.99:0/3884269928 conn(0x5589fbd4a2f0 msgr2=0x5589fbd50210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/3884269928 shutdown_connections 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.611+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/3884269928 wait complete. 2024-03-23T23:43:33.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 -- start start 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 Processor -- start 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd548e0 0x5589fbd5f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.612+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd66920 con 0x5589fbd4f770 2024-03-23T23:43:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd662c0 con 0x5589fbd4ed80 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1ebfa3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fbd66f80 con 0x5589fbd548e0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1ebfa3740 10 client.0 _ll_register_callbacks cb 0x7fb1d40f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34380/0 (socket says 172.21.15.99:34380) 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 -- 172.21.15.99:0/846271642 learned_addr learned my addr 172.21.15.99:0/846271642 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1d9400640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd548e0 0x5589fbd5f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 -- 172.21.15.99:0/846271642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd548e0 msgr2=0x5589fbd5f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd548e0 0x5589fbd5f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 -- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 msgr2=0x5589fbd5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 -- 172.21.15.99:0/846271642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589fbd5fb80 con 0x5589fbd4ed80 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1e2800640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc003650 tx=0x7fb1dc009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa62253979a06fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.613+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc02d040 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb1dc0051f0 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1cc003c70 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1dc045050 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 10 client.25226 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5589fbdc0e50 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc003c70 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb1dc018070 con 0x5589fbd4ed80 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 10 client.25226 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 20 client.25226 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbdc53e0 0x5589fbdc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 4 client.25226 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:43:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.614+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5589fbcbb520 con 0x5589fbdc53e0 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.615+0000 7fb1d9400640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbdc53e0 0x5589fbdc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.615+0000 7fb1d9400640 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbdc53e0 0x5589fbdc1080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fe15188a3deb68 server_cookie=67c9e0914e46cf9e in_seq=0 out_seq=0 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.615+0000 7fb1c7e00640 10 client.25226 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.615+0000 7fb1c7e00640 1 -- 172.21.15.99:0/846271642 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5589fbcbb520 con 0x5589fbdc53e0 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1c7e00640 10 client.25226 handle_command_reply: tid=0 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 client.25226 shutdown 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 msgr2=0x5589fbd5e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc003650 tx=0x7fb1dc009760 comp rx=0 tx=0).stop 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 shutdown_connections 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5589fbdc53e0 0x5589fbdc1080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589fbd548e0 0x5589fbd5f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5589fbd4f770 0x5589fbd5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1c7e00640 0 client.25226 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 --2- 172.21.15.99:0/846271642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589fbd4ed80 0x5589fbd5e990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 >> 172.21.15.99:0/846271642 conn(0x5589fbd4a2f0 msgr2=0x5589fbd57db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 shutdown_connections 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 1 -- 172.21.15.99:0/846271642 wait complete. 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.616+0000 7fb1ebfa3740 20 client.25226 trim_cache size 0 max 16384 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24565, 2024-03-23T23:43:33.620 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 139.922349567, 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 157.04000544100001 2024-03-23T23:43:33.621 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 msgr2=0x7fb1c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 0x7fb1c80a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40bd250 tx=0x7fb1d40f6530 comp rx=0 tx=0).stop 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 msgr2=0x7fb1e4199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4000f80 tx=0x7fb1d4032ec0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 shutdown_connections 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb1c80a2800 0x7fb1c80a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1e4105350 0x7fb1e41a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e4104990 0x7fb1e419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 --2- 172.21.15.99:0/2863258902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1e41037a0 0x7fb1e4199af0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.617+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 >> 172.21.15.99:0/2863258902 conn(0x7fb1e40fed10 msgr2=0x7fb1e4107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.618+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 shutdown_connections 2024-03-23T23:43:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.618+0000 7fb1e3200640 1 -- 172.21.15.99:0/2863258902 wait complete. 2024-03-23T23:43:33.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 -- 172.21.15.99:0/176988413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb440100760 msgr2=0x7fb440100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 --2- 172.21.15.99:0/176988413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb440100760 0x7fb440100bc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009900 tx=0x7fb42c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 -- 172.21.15.99:0/176988413 shutdown_connections 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 --2- 172.21.15.99:0/176988413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 0x7fb440101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 --2- 172.21.15.99:0/176988413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb440100760 0x7fb440100bc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 --2- 172.21.15.99:0/176988413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4400ffe00 0x7fb440100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.856+0000 7fb445800640 1 -- 172.21.15.99:0/176988413 >> 172.21.15.99:0/176988413 conn(0x7fb4400fba20 msgr2=0x7fb4400fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.857+0000 7fb445800640 1 -- 172.21.15.99:0/176988413 shutdown_connections 2024-03-23T23:43:33.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.857+0000 7fb445800640 1 -- 172.21.15.99:0/176988413 wait complete. 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 Processor -- start 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 -- start start 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb440100760 0x7fb440199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 0x7fb4401a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44010f440 con 0x7fb440101100 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44010f2c0 con 0x7fb440100760 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb445800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44010f5c0 con 0x7fb4400ffe00 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.858+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42154/0 (socket says 172.21.15.99:42154) 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 -- 172.21.15.99:0/2783183215 learned_addr learned my addr 172.21.15.99:0/2783183215 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43f400640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb440100760 0x7fb440199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb444e00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 0x7fb4401a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 -- 172.21.15.99:0/2783183215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb440100760 msgr2=0x7fb440199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb440100760 0x7fb440199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 -- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 msgr2=0x7fb4401a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 0x7fb4401a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 -- 172.21.15.99:0/2783183215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4401a1670 con 0x7fb4400ffe00 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb43000e9f0 tx=0x7fb43000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43cc00640 1 -- 172.21.15.99:0/2783183215 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000cd80 con 0x7fb4400ffe00 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43cc00640 1 -- 172.21.15.99:0/2783183215 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb430005500 con 0x7fb4400ffe00 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4401a1960 con 0x7fb4400ffe00 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.859+0000 7fb43cc00640 1 -- 172.21.15.99:0/2783183215 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430010690 con 0x7fb4400ffe00 2024-03-23T23:43:33.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.860+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4401a1e80 con 0x7fb4400ffe00 2024-03-23T23:43:33.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.862+0000 7fb449a72740 1 Processor -- start 2024-03-23T23:43:33.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 -- start start 2024-03-23T23:43:33.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afb770 0x55ad10b00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10a5fc90 con 0x55ad10afb770 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10a08980 con 0x55ad10b008e0 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb449a72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10a5bd30 con 0x55ad10afad80 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb444e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb43f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb444e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42168/0 (socket says 172.21.15.99:42168) 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb43f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34400/0 (socket says 172.21.15.99:34400) 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.863+0000 7fb444e00640 1 -- 172.21.15.99:0/2629783546 learned_addr learned my addr 172.21.15.99:0/2629783546 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 -- 172.21.15.99:0/2629783546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 msgr2=0x55ad10b02d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 -- 172.21.15.99:0/2629783546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afb770 msgr2=0x55ad10b00270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afb770 0x55ad10b00270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 -- 172.21.15.99:0/2629783546 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad10a20510 con 0x55ad10afad80 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb43f400640 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb43cc00640 1 -- 172.21.15.99:0/2783183215 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb4300046c0 con 0x7fb4400ffe00 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb444e00640 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb438009fc0 tx=0x7fb438010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5a1a1bed69977b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb435600640 1 -- 172.21.15.99:0/2629783546 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438003b70 con 0x55ad10afad80 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb435600640 1 -- 172.21.15.99:0/2629783546 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb438003d10 con 0x55ad10afad80 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.864+0000 7fb435600640 1 -- 172.21.15.99:0/2629783546 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb438011aa0 con 0x55ad10afad80 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 -- 172.21.15.99:0/2629783546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 msgr2=0x55ad10afb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb438009fc0 tx=0x7fb438010860 comp rx=0 tx=0).stop 2024-03-23T23:43:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 -- 172.21.15.99:0/2629783546 shutdown_connections 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb43cc00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 0x7fb4240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10b008e0 0x55ad10b02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afb770 0x55ad10b00270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 --2- 172.21.15.99:0/2629783546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10afb1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.865+0000 7fb449a72740 1 -- 172.21.15.99:0/2629783546 >> 172.21.15.99:0/2629783546 conn(0x55ad10af62f0 msgr2=0x55ad10af66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.866+0000 7fb43cc00640 1 -- 172.21.15.99:0/2783183215 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb4300cee90 con 0x7fb4400ffe00 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.866+0000 7fb449a72740 1 -- 172.21.15.99:0/2629783546 shutdown_connections 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.866+0000 7fb444e00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 0x7fb4240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.866+0000 7fb449a72740 1 -- 172.21.15.99:0/2629783546 wait complete. 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.866+0000 7fb449a72740 1 -- start start 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb444e00640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 0x7fb4240a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb438005fd0 tx=0x7fb438000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 Processor -- start 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b05840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b03890 con 0x55ad10b008e0 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10a1bdd0 con 0x55ad10afb770 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b03a10 con 0x55ad10afad80 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb449a72740 10 client.0 _ll_register_callbacks cb 0x55ad10773ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42180/0 (socket says 172.21.15.99:42180) 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb43fe00640 1 -- 172.21.15.99:0/2147596342 learned_addr learned my addr 172.21.15.99:0/2147596342 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb43f400640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b05840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.867+0000 7fb444e00640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 -- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 msgr2=0x55ad10b05840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b05840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 -- 172.21.15.99:0/2147596342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 msgr2=0x55ad10b0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 -- 172.21.15.99:0/2147596342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad107742d0 con 0x55ad10afad80 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43f400640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b05840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4300cc970 tx=0x7fb43009b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceac21a852fe58b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb42a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300b4060 con 0x55ad10afad80 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb4300a3b20 con 0x55ad10afad80 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb438003d10 con 0x55ad10afad80 2024-03-23T23:43:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb43009f960 con 0x55ad10afad80 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb449a72740 10 client.25267 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.868+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ad1083e6b0 con 0x55ad10afad80 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb438003d10 con 0x55ad10afad80 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb43009cc60 con 0x55ad10afad80 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 10 client.25267 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 10 client.25267 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 20 client.25267 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b11a40 0x55ad10b10c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 4 client.25267 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad10a01520 con 0x55ad10b11a40 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.869+0000 7fb43f400640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b11a40 0x55ad10b10c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.870+0000 7fb43f400640 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b11a40 0x55ad10b10c20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1029980af4d3e669 server_cookie=49488c58cecb2cd9 in_seq=0 out_seq=0 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.870+0000 7fb42a000640 10 client.25267 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb42a000640 1 -- 172.21.15.99:0/2147596342 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ad10a01520 con 0x55ad10b11a40 2024-03-23T23:43:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb42a000640 10 client.25267 handle_command_reply: tid=0 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 client.25267 shutdown 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 msgr2=0x55ad10b05300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4300cc970 tx=0x7fb43009b410 comp rx=0 tx=0).stop 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 shutdown_connections 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b11a40 0x55ad10b10c20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b05840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb42a000640 0 client.25267 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 --2- 172.21.15.99:0/2147596342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afad80 0x55ad10b05300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 >> 172.21.15.99:0/2147596342 conn(0x55ad10af62f0 msgr2=0x55ad10af54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 shutdown_connections 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.871+0000 7fb449a72740 1 -- 172.21.15.99:0/2147596342 wait complete. 2024-03-23T23:43:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.872+0000 7fb449a72740 20 client.25267 trim_cache size 0 max 16384 2024-03-23T23:43:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 Processor -- start 2024-03-23T23:43:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 -- start start 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afad80 0x55ad10af11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 0x55ad10b0c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b11260 con 0x55ad10b008e0 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b118c0 con 0x55ad10afad80 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.893+0000 7fb449a72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b11f20 con 0x55ad10afb770 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb444e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 0x55ad10b0c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb444e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39478/0 (socket says 172.21.15.99:39478) 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb444e00640 1 -- 172.21.15.99:0/2975835853 learned_addr learned my addr 172.21.15.99:0/2975835853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43fe00640 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afad80 0x55ad10af11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 -- 172.21.15.99:0/2975835853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afad80 msgr2=0x55ad10af11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afad80 0x55ad10af11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 -- 172.21.15.99:0/2975835853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 msgr2=0x55ad10b057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 -- 172.21.15.99:0/2975835853 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad10afbb50 con 0x55ad10afb770 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.894+0000 7fb43f400640 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 0x55ad10b0c810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51dc4aa363e09854 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb434c00640 1 -- 172.21.15.99:0/2975835853 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c005d50 con 0x55ad10afb770 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb434c00640 1 -- 172.21.15.99:0/2975835853 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42c04e030 con 0x55ad10afb770 2024-03-23T23:43:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb434c00640 1 -- 172.21.15.99:0/2975835853 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c005110 con 0x55ad10afb770 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 -- 172.21.15.99:0/2975835853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 msgr2=0x55ad10b0c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 0x55ad10b0c810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c009260 comp rx=0 tx=0).stop 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 -- 172.21.15.99:0/2975835853 shutdown_connections 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10b008e0 0x55ad10b057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10afb770 0x55ad10b0c810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 --2- 172.21.15.99:0/2975835853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afad80 0x55ad10af11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 -- 172.21.15.99:0/2975835853 >> 172.21.15.99:0/2975835853 conn(0x55ad10af62f0 msgr2=0x55ad10afc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.895+0000 7fb449a72740 1 -- 172.21.15.99:0/2975835853 shutdown_connections 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.896+0000 7fb449a72740 1 -- 172.21.15.99:0/2975835853 wait complete. 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.896+0000 7fb449a72740 1 -- start start 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.896+0000 7fb449a72740 1 Processor -- start 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afad80 0x55ad10b0c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10b008e0 0x55ad10b08b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b12be0 con 0x55ad10afad80 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b12580 con 0x55ad10afb770 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad10b13240 con 0x55ad10b008e0 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb449a72740 10 client.0 _ll_register_callbacks cb 0x7fb430009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afad80 0x55ad10b0c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb444e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10b008e0 0x55ad10b08b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:34434/0 (socket says 172.21.15.99:34434) 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 -- 172.21.15.99:0/893162900 learned_addr learned my addr 172.21.15.99:0/893162900 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 -- 172.21.15.99:0/893162900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10b008e0 msgr2=0x55ad10b08b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10b008e0 0x55ad10b08b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 -- 172.21.15.99:0/893162900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afad80 msgr2=0x55ad10b0c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afad80 0x55ad10b0c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 -- 172.21.15.99:0/893162900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad10b092b0 con 0x55ad10afb770 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb43f400640 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009850 tx=0x7fb42c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dcfde595ce06f61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.897+0000 7fb42a000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c004940 con 0x55ad10afb770 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb42c04e030 con 0x55ad10afb770 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb42c004020 con 0x55ad10afb770 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb42c004220 con 0x55ad10afb770 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 10 client.25244 fetch_fsmap learned FSMap version 26 2024-03-23T23:43:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ad10b6d700 con 0x55ad10afb770 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42c04e030 con 0x55ad10afb770 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb42c04a720 con 0x55ad10afb770 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 10 client.25244 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 20 client.25244 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b71950 0x55ad10b6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 4 client.25244 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.898+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ad10a6d1d0 con 0x55ad10b71950 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb43fe00640 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b71950 0x55ad10b6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb43fe00640 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b71950 0x55ad10b6d8d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53de63de0b64d4cc server_cookie=d8dcd8a232b53569 in_seq=0 out_seq=0 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb42a000640 10 client.25244 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb42a000640 1 -- 172.21.15.99:0/893162900 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ad10a6d1d0 con 0x55ad10b71950 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb42a000640 10 client.25244 handle_command_reply: tid=0 2024-03-23T23:43:33.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.899+0000 7fb449a72740 1 client.25244 shutdown 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 msgr2=0x55ad10b0c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009850 tx=0x7fb42c005de0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 shutdown_connections 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55ad10b71950 0x55ad10b6d8d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad10b008e0 0x55ad10b08b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad10afb770 0x55ad10b0c610 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 --2- 172.21.15.99:0/893162900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad10afad80 0x55ad10b0c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 >> 172.21.15.99:0/893162900 conn(0x55ad10af62f0 msgr2=0x55ad10b03de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb42a000640 0 client.25244 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 shutdown_connections 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 1 -- 172.21.15.99:0/893162900 wait complete. 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.900+0000 7fb449a72740 20 client.25244 trim_cache size 0 max 16384 2024-03-23T23:43:33.904 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "5fd03d00-e96d-11ee-95cd-87774f69a715", 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24569, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 143.24351370100001, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 26, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 87, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 150.54563849100001 2024-03-23T23:43:33.905 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 msgr2=0x7fb4240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 0x7fb4240a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb438005fd0 tx=0x7fb438000f40 comp rx=0 tx=0).stop 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 msgr2=0x7fb440199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb43000e9f0 tx=0x7fb43000edf0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 shutdown_connections 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb4240a2880 0x7fb4240a4d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb440101100 0x7fb4401a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb440100760 0x7fb440199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 --2- 172.21.15.99:0/2783183215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4400ffe00 0x7fb440199a50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 >> 172.21.15.99:0/2783183215 conn(0x7fb4400fba20 msgr2=0x7fb4400fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 shutdown_connections 2024-03-23T23:43:33.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:33.901+0000 7fb445800640 1 -- 172.21.15.99:0/2783183215 wait complete. 2024-03-23T23:43:33.919 INFO:teuthology.run_tasks:Running task kclient... 2024-03-23T23:43:33.944 INFO:tasks.kclient:Mounting kernel clients... 2024-03-23T23:43:33.944 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-23T23:43:33.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:33.944 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-23T23:43:33.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:33.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-23T23:43:34.100 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:33 smithi099 ceph-mon[31715]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:34.116 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-03-23T23:43:34.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:43:34.121 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-23T23:43:34.121 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T23:43:34.121 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:34.122 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T23:43:34.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:33 smithi149 ceph-mon[36083]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:34.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:33 smithi174 ceph-mon[37002]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:43:34.564 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:43:36.298 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:35 smithi099 ceph-mon[31715]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:36.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:35 smithi149 ceph-mon[36083]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:36.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:35 smithi174 ceph-mon[37002]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:43:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 -- 172.21.15.99:0/3894668532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 msgr2=0x7f02741085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 --2- 172.21.15.99:0/3894668532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f02741085c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0268009990 tx=0x7f0268009ca0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 -- 172.21.15.99:0/3894668532 shutdown_connections 2024-03-23T23:43:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 --2- 172.21.15.99:0/3894668532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 0x7f027410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 --2- 172.21.15.99:0/3894668532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f02741085c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 --2- 172.21.15.99:0/3894668532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f027406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 -- 172.21.15.99:0/3894668532 >> 172.21.15.99:0/3894668532 conn(0x7f0274077fc0 msgr2=0x7f02740783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.846+0000 7f027ba00700 1 -- 172.21.15.99:0/3894668532 shutdown_connections 2024-03-23T23:43:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.847+0000 7f027ba00700 1 -- 172.21.15.99:0/3894668532 wait complete. 2024-03-23T23:43:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.848+0000 7f027ba00700 1 Processor -- start 2024-03-23T23:43:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.848+0000 7f027ba00700 1 -- start start 2024-03-23T23:43:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 0x7f02741a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f027419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274112470 con 0x7f027406a7e0 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741122f0 con 0x7f02741081e0 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f027ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741125f0 con 0x7f0274069e40 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42238/0 (socket says 172.21.15.99:42238) 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 -- 172.21.15.99:0/3501766818 learned_addr learned my addr 172.21.15.99:0/3501766818 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279a00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f027419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0273e00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 0x7f02741a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:36.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 -- 172.21.15.99:0/3501766818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 msgr2=0x7f027419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f027419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 -- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 msgr2=0x7f02741a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.849+0000 7f0279000700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 0x7f02741a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f0279000700 1 -- 172.21.15.99:0/3501766818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027419b050 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f0279000700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268027070 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02680056b0 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027419b2e0 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268017650 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.850+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741a7600 con 0x7f0274069e40 2024-03-23T23:43:36.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.851+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274068e50 con 0x7f0274069e40 2024-03-23T23:43:36.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.855+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f026802a070 con 0x7f0274069e40 2024-03-23T23:43:36.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.856+0000 7f0271600700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 0x7f025c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:36.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.856+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f02680cf800 con 0x7f0274069e40 2024-03-23T23:43:36.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.856+0000 7f0273e00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 0x7f025c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:36.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.857+0000 7f0273e00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 0x7f025c0a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0260009680 tx=0x7f0260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:36.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:36.860+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0268013070 con 0x7f0274069e40 2024-03-23T23:43:37.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.137+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f02741a77d0 con 0x7f0274069e40 2024-03-23T23:43:37.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.138+0000 7f0271600700 1 -- 172.21.15.99:0/3501766818 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f026809c8b0 con 0x7f0274069e40 2024-03-23T23:43:37.138 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T23:43:37.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 msgr2=0x7f025c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:37.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 0x7f025c0a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0260009680 tx=0x7f0260009260 comp rx=0 tx=0).stop 2024-03-23T23:43:37.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 msgr2=0x7f02741a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:37.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268005590 comp rx=0 tx=0).stop 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 shutdown_connections 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f025c0a2910 0x7f025c0a4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02741081e0 0x7f027419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.140+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0274069e40 0x7f02741a0730 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.141+0000 7f027ba00700 1 --2- 172.21.15.99:0/3501766818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f027406a7e0 0x7f02741a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:37.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.141+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 >> 172.21.15.99:0/3501766818 conn(0x7f0274077fc0 msgr2=0x7f027410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:37.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.141+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 shutdown_connections 2024-03-23T23:43:37.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:37.141+0000 7f027ba00700 1 -- 172.21.15.99:0/3501766818 wait complete. 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T23:43:37.939 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T23:43:37.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:37.939 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T23:43:37.988 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41942sec preferred_lft 41942sec 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-23T23:43:37.989 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-23T23:43:37.989 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T23:43:37.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T23:43:37.990 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-23T23:43:38.321 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:37 smithi099 ceph-mon[31715]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:38.321 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:37 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3501766818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T23:43:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:37 smithi149 ceph-mon[36083]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:37 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3501766818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T23:43:38.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:37 smithi174 ceph-mon[37002]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:38.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:37 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3501766818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T23:43:38.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:38.450 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T23:43:38.530 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-23T23:43:38.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:38.535 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T23:43:38.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T23:43:38.591 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-23T23:43:39.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:39.251 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-23T23:43:39.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:39.300 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-03-23T23:43:39.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:39.340 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-23T23:43:39.340 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-23T23:43:39.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T23:43:39.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T23:43:39.340 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-23T23:43:39.520 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T23:43:39.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:39.520 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-23T23:43:39.520 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-23T23:43:39.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T23:43:39.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T23:43:39.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T23:43:39.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T23:43:39.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T23:43:39.521 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-23T23:43:39.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:39.798 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-23T23:43:39.798 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-23T23:43:39.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-23T23:43:39.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T23:43:39.798 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-23T23:43:39.951 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T23:43:39.951 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:39.993 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T23:43:39.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T23:43:39.994 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:40.076 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-23T23:43:40.076 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-23T23:43:40.076 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/b5b8b0ac-01ee-424f-ab5e-18f6d295a675 2024-03-23T23:43:40.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:40.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/b5b8b0ac-01ee-424f-ab5e-18f6d295a675 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-23T23:43:40.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:39 smithi099 ceph-mon[31715]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:40.233 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-23T23:43:40.233 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-03-23T23:43:40.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:39 smithi149 ceph-mon[36083]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:40.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:39 smithi174 ceph-mon[37002]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-23T23:43:40.858 INFO:teuthology.orchestra.run.smithi099.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-23T23:43:40.858 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-03-23T23:43:40.859 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: resolved to: "172.21.15.99:3300,172.21.15.149:3300,172.21.15.174:3300" 2024-03-23T23:43:40.859 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: trying mount with new device syntax: 0@5fd03d00-e96d-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/b5b8b0ac-01ee-424f-ab5e-18f6d295a675 2024-03-23T23:43:40.859 INFO:teuthology.orchestra.run.smithi099.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.99:3300/172.21.15.149:3300/172.21.15.174:3300" will pass to kernel 2024-03-23T23:43:40.863 INFO:teuthology.orchestra.run.smithi099.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-23T23:43:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: You just mounted a file system that supports labels which does not 2024-03-23T23:43:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T23:43:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T23:43:40.863 INFO:teuthology.orchestra.run.smithi099.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T23:43:40.866 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T23:43:40.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:40.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:40.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:40.954 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T23:43:41.001 INFO:teuthology.orchestra.run.smithi099.stdout:client25253 2024-03-23T23:43:41.002 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:43:41.002 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T23:43:41.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:41.092 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T23:43:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:client25253 2024-03-23T23:43:41.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:41.185 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T23:43:41.229 INFO:teuthology.orchestra.run.smithi099.stdout:client25253 2024-03-23T23:43:41.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:41.230 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715.client25253/status') 2024-03-23T23:43:41.347 INFO:teuthology.orchestra.run.smithi099.stdout:instance: client.25253 (3)172.21.15.99:0/1679431445 2024-03-23T23:43:41.347 INFO:teuthology.orchestra.run.smithi099.stdout:blocklisted: false 2024-03-23T23:43:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records in 2024-03-23T23:43:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:0+1 records out 2024-03-23T23:43:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:71 bytes copied, 6.481e-05 s, 1.1 MB/s 2024-03-23T23:43:41.351 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T23:43:41.361 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T23:43:41.361 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:43:41.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.545+0000 7fbadfe00640 1 -- 172.21.15.99:0/2678409918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8108ef0 msgr2=0x7fbad8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.545+0000 7fbadfe00640 1 --2- 172.21.15.99:0/2678409918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8108ef0 0x7fbad8109370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009900 tx=0x7fbac402bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 -- 172.21.15.99:0/2678409918 shutdown_connections 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 --2- 172.21.15.99:0/2678409918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad81098b0 0x7fbad8110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 --2- 172.21.15.99:0/2678409918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8108ef0 0x7fbad8109370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 --2- 172.21.15.99:0/2678409918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad8107d00 0x7fbad8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 -- 172.21.15.99:0/2678409918 >> 172.21.15.99:0/2678409918 conn(0x7fbad8076cb0 msgr2=0x7fbad80790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:41.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 -- 172.21.15.99:0/2678409918 shutdown_connections 2024-03-23T23:43:41.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.546+0000 7fbadfe00640 1 -- 172.21.15.99:0/2678409918 wait complete. 2024-03-23T23:43:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 Processor -- start 2024-03-23T23:43:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 -- start start 2024-03-23T23:43:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 0x7fbad819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad81098b0 0x7fbad81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad81139d0 con 0x7fbad8108ef0 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8113850 con 0x7fbad8107d00 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.547+0000 7fbadfe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8113b50 con 0x7fbad81098b0 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadde00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad81098b0 0x7fbad81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42234/0 (socket says 172.21.15.99:42234) 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 -- 172.21.15.99:0/1560539207 learned_addr learned my addr 172.21.15.99:0/1560539207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadca00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 0x7fbad819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 -- 172.21.15.99:0/1560539207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad81098b0 msgr2=0x7fbad81a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad81098b0 0x7fbad81a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 -- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 msgr2=0x7fbad819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 0x7fbad819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 -- 172.21.15.99:0/1560539207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad81a5b70 con 0x7fbad8107d00 2024-03-23T23:43:41.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadca00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 0x7fbad819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:41.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.548+0000 7fbadd400640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e9c0 tx=0x7fbacc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:41.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.549+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00cd80 con 0x7fbad8107d00 2024-03-23T23:43:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.549+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbacc0035f0 con 0x7fbad8107d00 2024-03-23T23:43:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.549+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc010690 con 0x7fbad8107d00 2024-03-23T23:43:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.549+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad81a5e60 con 0x7fbad8107d00 2024-03-23T23:43:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.549+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a62c0 con 0x7fbad8107d00 2024-03-23T23:43:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.550+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad810ef50 con 0x7fbad8107d00 2024-03-23T23:43:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.552+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbacc0037a0 con 0x7fbad8107d00 2024-03-23T23:43:41.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.553+0000 7fbad6000640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 0x7fbac00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:41.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.553+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbacc0cee60 con 0x7fbad8107d00 2024-03-23T23:43:41.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.553+0000 7fbadca00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 0x7fbac00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:41.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.554+0000 7fbadca00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 0x7fbac00a4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbac402c240 tx=0x7fbac402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:41.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.557+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbacc09be80 con 0x7fbad8107d00 2024-03-23T23:43:41.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.909+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fbad80623e0 con 0x7fbad8107d00 2024-03-23T23:43:41.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.971+0000 7fbad6000640 1 -- 172.21.15.99:0/1560539207 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7fbacc09b780 con 0x7fbad8107d00 2024-03-23T23:43:41.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 msgr2=0x7fbac00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:41.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 0x7fbac00a4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbac402c240 tx=0x7fbac402df90 comp rx=0 tx=0).stop 2024-03-23T23:43:41.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 msgr2=0x7fbad819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:41.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e9c0 tx=0x7fbacc00ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 shutdown_connections 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac00a2920 0x7fbac00a4de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad81098b0 0x7fbad81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8108ef0 0x7fbad819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 --2- 172.21.15.99:0/1560539207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad8107d00 0x7fbad819df50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 >> 172.21.15.99:0/1560539207 conn(0x7fbad8076cb0 msgr2=0x7fbad810c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 shutdown_connections 2024-03-23T23:43:41.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:41.975+0000 7fbadfe00640 1 -- 172.21.15.99:0/1560539207 wait complete. 2024-03-23T23:43:41.993 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-23T23:43:42.003 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-23T23:43:42.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 -- 172.21.15.99:0/2069155133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141016c0 msgr2=0x7f0a14101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 --2- 172.21.15.99:0/2069155133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141016c0 0x7f0a14101ae0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009900 tx=0x7f0a0802bc50 comp rx=0 tx=0).stop 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 -- 172.21.15.99:0/2069155133 shutdown_connections 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 --2- 172.21.15.99:0/2069155133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 0x7f0a14109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 --2- 172.21.15.99:0/2069155133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141028b0 0x7f0a14102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.221+0000 7f0a1c800640 1 --2- 172.21.15.99:0/2069155133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141016c0 0x7f0a14101ae0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.222+0000 7f0a1c800640 1 -- 172.21.15.99:0/2069155133 >> 172.21.15.99:0/2069155133 conn(0x7f0a140fcc50 msgr2=0x7f0a140ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.222+0000 7f0a1c800640 1 -- 172.21.15.99:0/2069155133 shutdown_connections 2024-03-23T23:43:42.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.222+0000 7f0a1c800640 1 -- 172.21.15.99:0/2069155133 wait complete. 2024-03-23T23:43:42.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.223+0000 7f0a1c800640 1 Processor -- start 2024-03-23T23:43:42.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.223+0000 7f0a1c800640 1 -- start start 2024-03-23T23:43:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 0x7f0a14197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 0x7f0a1419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1410d390 con 0x7f0a141016c0 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1410d210 con 0x7f0a14103270 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1c800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1410d510 con 0x7f0a141028b0 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a19400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 0x7f0a14197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a19e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55356/0 (socket says 172.21.15.99:55356) 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a19e00640 1 -- 172.21.15.99:0/1449886185 learned_addr learned my addr 172.21.15.99:0/1449886185 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a1a800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 0x7f0a1419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.224+0000 7f0a19400640 1 -- 172.21.15.99:0/1449886185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 msgr2=0x7f0a1419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19400640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 0x7f0a1419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19400640 1 -- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 msgr2=0x7f0a141979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19400640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19400640 1 -- 172.21.15.99:0/1449886185 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1419f5c0 con 0x7f0a141028b0 2024-03-23T23:43:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19e00640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a19400640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 0x7f0a14197ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e9f0 tx=0x7f0a0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400ccc0 con 0x7f0a141028b0 2024-03-23T23:43:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a04010910 con 0x7f0a141028b0 2024-03-23T23:43:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04018960 con 0x7f0a141028b0 2024-03-23T23:43:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1419f8b0 con 0x7f0a141028b0 2024-03-23T23:43:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.225+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1419fdb0 con 0x7f0a141028b0 2024-03-23T23:43:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.227+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a141089e0 con 0x7f0a141028b0 2024-03-23T23:43:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.229+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0a04010ac0 con 0x7f0a141028b0 2024-03-23T23:43:42.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.230+0000 7f0a0ea00640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 0x7f09fc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.230+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0a040cedf0 con 0x7f0a141028b0 2024-03-23T23:43:42.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.230+0000 7f0a19e00640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 0x7f09fc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.231+0000 7f0a19e00640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 0x7f09fc0a4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a0802c900 tx=0x7f0a08000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:42.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.234+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a0409b7c0 con 0x7f0a141028b0 2024-03-23T23:43:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:41 smithi099 ceph-mon[31715]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:41 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1560539207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:41 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:41 smithi149 ceph-mon[36083]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:41 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1560539207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:41 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:41 smithi174 ceph-mon[37002]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:43:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:41 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1560539207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:41 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-23T23:43:42.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.551+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a140622a0 con 0x7f0a141028b0 2024-03-23T23:43:42.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.553+0000 7f0a0ea00640 1 -- 172.21.15.99:0/1449886185 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f0a0409f860 con 0x7f0a141028b0 2024-03-23T23:43:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:42.552 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:42.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 msgr2=0x7f09fc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 0x7f09fc0a4d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a0802c900 tx=0x7f0a08000bc0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 msgr2=0x7f0a14197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 0x7f0a14197ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e9f0 tx=0x7f0a0400eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 shutdown_connections 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f09fc0a2880 0x7f09fc0a4d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14103270 0x7f0a1419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a141028b0 0x7f0a14197ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 --2- 172.21.15.99:0/1449886185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a141016c0 0x7f0a141979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 >> 172.21.15.99:0/1449886185 conn(0x7f0a140fcc50 msgr2=0x7f0a14105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 shutdown_connections 2024-03-23T23:43:42.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.555+0000 7f0a1c800640 1 -- 172.21.15.99:0/1449886185 wait complete. 2024-03-23T23:43:42.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:43:42.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.783+0000 7f796e000640 1 -- 172.21.15.99:0/528029451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 msgr2=0x7f796810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.783+0000 7f796e000640 1 --2- 172.21.15.99:0/528029451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f796810bd20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7960009860 tx=0x7f796002bb70 comp rx=0 tx=0).stop 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 -- 172.21.15.99:0/528029451 shutdown_connections 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 --2- 172.21.15.99:0/528029451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f796810bd20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 --2- 172.21.15.99:0/528029451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7968104ad0 0x7f7968104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 --2- 172.21.15.99:0/528029451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79681038e0 0x7f7968103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 -- 172.21.15.99:0/528029451 >> 172.21.15.99:0/528029451 conn(0x7f79680fee50 msgr2=0x7f7968101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:42.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 -- 172.21.15.99:0/528029451 shutdown_connections 2024-03-23T23:43:42.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.784+0000 7f796e000640 1 -- 172.21.15.99:0/528029451 wait complete. 2024-03-23T23:43:42.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.785+0000 7f796e000640 1 Processor -- start 2024-03-23T23:43:42.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.785+0000 7f796e000640 1 -- start start 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7968104ad0 0x7f796819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f79681a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810f540 con 0x7f7968104ad0 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810f3c0 con 0x7f7968105490 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f796e000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810f6c0 con 0x7f79681038e0 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f7967400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f7967400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57320/0 (socket says 172.21.15.99:57320) 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f7967400640 1 -- 172.21.15.99:0/251774597 learned_addr learned my addr 172.21.15.99:0/251774597 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:42.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f795ea00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7968104ad0 0x7f796819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.786+0000 7f7967e00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f79681a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 -- 172.21.15.99:0/251774597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 msgr2=0x7f79681a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f79681a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 -- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7968104ad0 msgr2=0x7f796819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7968104ad0 0x7f796819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 -- 172.21.15.99:0/251774597 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79681a17a0 con 0x7f79681038e0 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967e00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f79681a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:42.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7967400640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795800ebc0 tx=0x7f795800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958003b70 con 0x7f79681038e0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79681a1a90 con 0x7f79681038e0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7958003d10 con 0x7f79681038e0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.787+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958019980 con 0x7f79681038e0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.788+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79681a1fe0 con 0x7f79681038e0 2024-03-23T23:43:42.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.789+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796810ac00 con 0x7f79681038e0 2024-03-23T23:43:42.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.792+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7958004b60 con 0x7f79681038e0 2024-03-23T23:43:42.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.794+0000 7f7964c00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:42.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.794+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f79580cf260 con 0x7f79681038e0 2024-03-23T23:43:42.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.794+0000 7f795ea00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:42.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.795+0000 7f795ea00640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 0x7f794c0a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f796819b090 tx=0x7f7950009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:42.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:42.797+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f795809c2e0 con 0x7f79681038e0 2024-03-23T23:43:43.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.023+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f796802d2d0 con 0x7f79681038e0 2024-03-23T23:43:43.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.024+0000 7f7964c00640 1 -- 172.21.15.99:0/251774597 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f795809bbe0 con 0x7f79681038e0 2024-03-23T23:43:43.025 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:43:43.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 msgr2=0x7f794c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 0x7f794c0a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f796819b090 tx=0x7f7950009260 comp rx=0 tx=0).stop 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 msgr2=0x7f7968199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f795800ebc0 tx=0x7f795800c6a0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 shutdown_connections 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7968105490 0x7f79681a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7968104ad0 0x7f796819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 --2- 172.21.15.99:0/251774597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79681038e0 0x7f7968199b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 >> 172.21.15.99:0/251774597 conn(0x7f79680fee50 msgr2=0x7f7968107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 shutdown_connections 2024-03-23T23:43:43.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.028+0000 7f796e000640 1 -- 172.21.15.99:0/251774597 wait complete. 2024-03-23T23:43:43.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:43:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 -- 172.21.15.99:0/893718283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 msgr2=0x7fdb64073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 --2- 172.21.15.99:0/893718283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64073ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009ca0 tx=0x7fdb5402c180 comp rx=0 tx=0).stop 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 -- 172.21.15.99:0/893718283 shutdown_connections 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 --2- 172.21.15.99:0/893718283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb64109430 0x7fdb6410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 --2- 172.21.15.99:0/893718283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb640740a0 0x7fdb64074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 --2- 172.21.15.99:0/893718283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64073ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 -- 172.21.15.99:0/893718283 >> 172.21.15.99:0/893718283 conn(0x7fdb640fcb10 msgr2=0x7fdb640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 -- 172.21.15.99:0/893718283 shutdown_connections 2024-03-23T23:43:43.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.260+0000 7fdb68a00640 1 -- 172.21.15.99:0/893718283 wait complete. 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.261+0000 7fdb68a00640 1 Processor -- start 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.261+0000 7fdb68a00640 1 -- start start 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb64109430 0x7fdb641a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410f3f0 con 0x7fdb64109430 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410f270 con 0x7fdb64075680 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb68a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410f570 con 0x7fdb640740a0 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb62800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb59e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb62800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57334/0 (socket says 172.21.15.99:57334) 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb59e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42284/0 (socket says 172.21.15.99:42284) 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb62800640 1 -- 172.21.15.99:0/3578988184 learned_addr learned my addr 172.21.15.99:0/3578988184 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.262+0000 7fdb63200640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb64109430 0x7fdb641a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 -- 172.21.15.99:0/3578988184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 msgr2=0x7fdb64199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 -- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb64109430 msgr2=0x7fdb641a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb64109430 0x7fdb641a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 -- 172.21.15.99:0/3578988184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb641a16b0 con 0x7fdb640740a0 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb62800640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdb540326d0 tx=0x7fdb54035f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:43.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb54039040 con 0x7fdb640740a0 2024-03-23T23:43:43.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb54044070 con 0x7fdb640740a0 2024-03-23T23:43:43.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb641a1940 con 0x7fdb640740a0 2024-03-23T23:43:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.263+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb540043a0 con 0x7fdb640740a0 2024-03-23T23:43:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.264+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb641a1e50 con 0x7fdb640740a0 2024-03-23T23:43:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.265+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb641081f0 con 0x7fdb640740a0 2024-03-23T23:43:43.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.267+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdb54004540 con 0x7fdb640740a0 2024-03-23T23:43:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.268+0000 7fdb5be00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 0x7fdb480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.268+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdb540f0fc0 con 0x7fdb640740a0 2024-03-23T23:43:43.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.268+0000 7fdb59e00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 0x7fdb480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.269+0000 7fdb59e00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 0x7fdb480a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb6419afa0 tx=0x7fdb4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:43.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.272+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb540bdfe0 con 0x7fdb640740a0 2024-03-23T23:43:43.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:42 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T23:43:43.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:42 smithi099 ceph-mon[31715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:43:43.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:42 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1449886185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:42 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:42 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T23:43:43.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:42 smithi149 ceph-mon[36083]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:43:43.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:42 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1449886185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:42 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:42 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-23T23:43:43.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:42 smithi174 ceph-mon[37002]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-23T23:43:43.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:42 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1449886185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:42 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:43:43.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.552+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb640622a0 con 0x7fdb640740a0 2024-03-23T23:43:43.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.553+0000 7fdb5be00640 1 -- 172.21.15.99:0/3578988184 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb540bd8e0 con 0x7fdb640740a0 2024-03-23T23:43:43.552 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:43:43.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 msgr2=0x7fdb480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 0x7fdb480a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb6419afa0 tx=0x7fdb4c005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 msgr2=0x7fdb64199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdb540326d0 tx=0x7fdb54035f60 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 shutdown_connections 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdb480a2880 0x7fdb480a4d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb64109430 0x7fdb641a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb64075680 0x7fdb64199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 --2- 172.21.15.99:0/3578988184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb640740a0 0x7fdb64199a90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 >> 172.21.15.99:0/3578988184 conn(0x7fdb640fcb10 msgr2=0x7fdb641053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 shutdown_connections 2024-03-23T23:43:43.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.556+0000 7fdb68a00640 1 -- 172.21.15.99:0/3578988184 wait complete. 2024-03-23T23:43:43.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:43:43.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 -- 172.21.15.99:0/2689620085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb0103150 msgr2=0x7f4cb01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 --2- 172.21.15.99:0/2689620085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb0103150 0x7f4cb01099e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009ca0 tx=0x7f4ca802c100 comp rx=0 tx=0).stop 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 -- 172.21.15.99:0/2689620085 shutdown_connections 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 --2- 172.21.15.99:0/2689620085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb0103150 0x7f4cb01099e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 --2- 172.21.15.99:0/2689620085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0102790 0x7f4cb0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 --2- 172.21.15.99:0/2689620085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb01015a0 0x7f4cb01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 -- 172.21.15.99:0/2689620085 >> 172.21.15.99:0/2689620085 conn(0x7f4cb00fcb10 msgr2=0x7f4cb00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 -- 172.21.15.99:0/2689620085 shutdown_connections 2024-03-23T23:43:43.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.794+0000 7f4caf200640 1 -- 172.21.15.99:0/2689620085 wait complete. 2024-03-23T23:43:43.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.795+0000 7f4caf200640 1 Processor -- start 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.795+0000 7f4caf200640 1 -- start start 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb0102790 0x7f4cb0197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0103150 0x7f4cb019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb010d230 con 0x7f4cb0103150 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb010d0b0 con 0x7f4cb0102790 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4caf200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb010d3b0 con 0x7f4cb01015a0 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57352/0 (socket says 172.21.15.99:57352) 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 -- 172.21.15.99:0/2856546565 learned_addr learned my addr 172.21.15.99:0/2856546565 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:43.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cad400640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb0102790 0x7f4cb0197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cae800640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0103150 0x7f4cb019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 -- 172.21.15.99:0/2856546565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb0102790 msgr2=0x7f4cb0197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb0102790 0x7f4cb0197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 -- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0103150 msgr2=0x7f4cb019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0103150 0x7f4cb019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.796+0000 7f4cade00640 1 -- 172.21.15.99:0/2856546565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb019f410 con 0x7f4cb01015a0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4cade00640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000e9c0 tx=0x7f4ca000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca000cd80 con 0x7f4cb01015a0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ca00035f0 con 0x7f4cb01015a0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb019f700 con 0x7f4cb01015a0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0010690 con 0x7f4cb01015a0 2024-03-23T23:43:43.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.797+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb019fc50 con 0x7f4cb01015a0 2024-03-23T23:43:43.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.799+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb01088c0 con 0x7f4cb01015a0 2024-03-23T23:43:43.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.800+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4ca00037a0 con 0x7f4cb01015a0 2024-03-23T23:43:43.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.802+0000 7f4ca6a00640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 0x7f4c940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:43.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.803+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ca00cee60 con 0x7f4cb01015a0 2024-03-23T23:43:43.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.803+0000 7f4cad400640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 0x7f4c940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:43.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.803+0000 7f4cad400640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 0x7f4c940a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0198d00 tx=0x7f4c9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:43.810+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ca009be80 con 0x7f4cb01015a0 2024-03-23T23:43:44.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.044+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cb002d070 con 0x7f4cb01015a0 2024-03-23T23:43:44.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.045+0000 7f4ca6a00640 1 -- 172.21.15.99:0/2856546565 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cb002d070 con 0x7f4cb01015a0 2024-03-23T23:43:44.044 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:43:44.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 msgr2=0x7f4c940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 0x7f4c940a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0198d00 tx=0x7f4c9c005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:44.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 msgr2=0x7f4cb01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000e9c0 tx=0x7f4ca000ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 shutdown_connections 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4c940a2880 0x7f4c940a4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4cb0103150 0x7f4cb019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cb0102790 0x7f4cb0197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 --2- 172.21.15.99:0/2856546565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cb01015a0 0x7f4cb01977f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 >> 172.21.15.99:0/2856546565 conn(0x7f4cb00fcb10 msgr2=0x7f4cb0105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 shutdown_connections 2024-03-23T23:43:44.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.049+0000 7f4caf200640 1 -- 172.21.15.99:0/2856546565 wait complete. 2024-03-23T23:43:44.069 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:44.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T23:43:44.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/251774597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:43 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3578988184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 -- 172.21.15.99:0/3161368675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 msgr2=0x7f3a34104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3161368675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a34104f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009900 tx=0x7f3a2002bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 -- 172.21.15.99:0/3161368675 shutdown_connections 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3161368675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a3410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3161368675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a34104f50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3161368675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a341038e0 0x7f3a34103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 -- 172.21.15.99:0/3161368675 >> 172.21.15.99:0/3161368675 conn(0x7f3a340fee90 msgr2=0x7f3a341012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 -- 172.21.15.99:0/3161368675 shutdown_connections 2024-03-23T23:43:44.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.277+0000 7f3a3a000640 1 -- 172.21.15.99:0/3161368675 wait complete. 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.278+0000 7f3a3a000640 1 Processor -- start 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.278+0000 7f3a3a000640 1 -- start start 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a341038e0 0x7f3a34199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a341a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410f590 con 0x7f3a341038e0 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410f410 con 0x7f3a34104ad0 2024-03-23T23:43:44.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a3a000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410f710 con 0x7f3a34105490 2024-03-23T23:43:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a32a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42302/0 (socket says 172.21.15.99:42302) 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a32a00640 1 -- 172.21.15.99:0/3633003701 learned_addr learned my addr 172.21.15.99:0/3633003701 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.279+0000 7f3a33e00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a341a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 -- 172.21.15.99:0/3633003701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 msgr2=0x7f3a341a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a341a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 -- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a341038e0 msgr2=0x7f3a34199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a341038e0 0x7f3a34199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 -- 172.21.15.99:0/3633003701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a341a17f0 con 0x7f3a34104ad0 2024-03-23T23:43:44.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a33e00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a341a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.280+0000 7f3a32a00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000c00 tx=0x7f3a2002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.282+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20043020 con 0x7f3a34104ad0 2024-03-23T23:43:44.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.282+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a200054e0 con 0x7f3a34104ad0 2024-03-23T23:43:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.282+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20003c40 con 0x7f3a34104ad0 2024-03-23T23:43:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.282+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a341a1a80 con 0x7f3a34104ad0 2024-03-23T23:43:44.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.282+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a341a1f80 con 0x7f3a34104ad0 2024-03-23T23:43:44.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.284+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3410ac00 con 0x7f3a34104ad0 2024-03-23T23:43:44.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.286+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3a20046020 con 0x7f3a34104ad0 2024-03-23T23:43:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.288+0000 7f3a2be00640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 0x7f3a180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.288+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3a200f0af0 con 0x7f3a34104ad0 2024-03-23T23:43:44.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.289+0000 7f3a33400640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 0x7f3a180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.290+0000 7f3a33400640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 0x7f3a180a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005d00 tx=0x7f3a24005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:44.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.291+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a2004d020 con 0x7f3a34104ad0 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/251774597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:43 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3578988184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/251774597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:43 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3578988184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:44.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.530+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a340622a0 con 0x7f3a34104ad0 2024-03-23T23:43:44.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.532+0000 7f3a2be00640 1 -- 172.21.15.99:0/3633003701 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f3a200bdb70 con 0x7f3a34104ad0 2024-03-23T23:43:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:44.532 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:44.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 msgr2=0x7f3a180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 0x7f3a180a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005d00 tx=0x7f3a24005c50 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 msgr2=0x7f3a3419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000c00 tx=0x7f3a2002c650 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 shutdown_connections 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3a180a2880 0x7f3a180a4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a34105490 0x7f3a341a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a34104ad0 0x7f3a3419a110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 --2- 172.21.15.99:0/3633003701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a341038e0 0x7f3a34199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.534+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 >> 172.21.15.99:0/3633003701 conn(0x7f3a340fee90 msgr2=0x7f3a34107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.535+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 shutdown_connections 2024-03-23T23:43:44.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.535+0000 7f3a3a000640 1 -- 172.21.15.99:0/3633003701 wait complete. 2024-03-23T23:43:44.554 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:43:44.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.746+0000 7febdc800640 1 -- 172.21.15.99:0/2732399354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 msgr2=0x7febd410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.746+0000 7febdc800640 1 --2- 172.21.15.99:0/2732399354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd410bd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febcc009900 tx=0x7febcc02bc40 comp rx=0 tx=0).stop 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 -- 172.21.15.99:0/2732399354 shutdown_connections 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 --2- 172.21.15.99:0/2732399354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd410bd20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 --2- 172.21.15.99:0/2732399354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd4104ad0 0x7febd4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 --2- 172.21.15.99:0/2732399354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd41038e0 0x7febd4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 -- 172.21.15.99:0/2732399354 >> 172.21.15.99:0/2732399354 conn(0x7febd40fee50 msgr2=0x7febd4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 -- 172.21.15.99:0/2732399354 shutdown_connections 2024-03-23T23:43:44.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.747+0000 7febdc800640 1 -- 172.21.15.99:0/2732399354 wait complete. 2024-03-23T23:43:44.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.748+0000 7febdc800640 1 Processor -- start 2024-03-23T23:43:44.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.748+0000 7febdc800640 1 -- start start 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd41a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f590 con 0x7febd41038e0 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f410 con 0x7febd4104ad0 2024-03-23T23:43:44.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febdc800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f710 con 0x7febd4105490 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd41a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42328/0 (socket says 172.21.15.99:42328) 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55436/0 (socket says 172.21.15.99:55436) 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9400640 1 -- 172.21.15.99:0/2003628255 learned_addr learned my addr 172.21.15.99:0/2003628255 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 -- 172.21.15.99:0/2003628255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 msgr2=0x7febd419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 -- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 msgr2=0x7febd4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febda800640 1 -- 172.21.15.99:0/2003628255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd41a1770 con 0x7febd4105490 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9e00640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:44.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.749+0000 7febd9400640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:44.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febda800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd41a1100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febcc02c180 tx=0x7febcc02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:44.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc041070 con 0x7febd4105490 2024-03-23T23:43:44.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febcc004450 con 0x7febd4105490 2024-03-23T23:43:44.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc003e00 con 0x7febd4105490 2024-03-23T23:43:44.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd41a1a00 con 0x7febd4105490 2024-03-23T23:43:44.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.750+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41a1f10 con 0x7febd4105490 2024-03-23T23:43:44.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.751+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd410ab30 con 0x7febd4105490 2024-03-23T23:43:44.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.753+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7febcc044070 con 0x7febd4105490 2024-03-23T23:43:44.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.754+0000 7febcaa00640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 0x7febc00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:44.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.754+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7febcc0f0860 con 0x7febd4105490 2024-03-23T23:43:44.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.755+0000 7febd9e00640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 0x7febc00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:44.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.755+0000 7febd9e00640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 0x7febc00a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7febd4104930 tx=0x7febd0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:44.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:44.756+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febcc0bd2c0 con 0x7febd4105490 2024-03-23T23:43:45.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.013+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febd402d200 con 0x7febd4105490 2024-03-23T23:43:45.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.014+0000 7febcaa00640 1 -- 172.21.15.99:0/2003628255 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7febcc0c1360 con 0x7febd4105490 2024-03-23T23:43:45.014 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 msgr2=0x7febc00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 0x7febc00a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7febd4104930 tx=0x7febd0009500 comp rx=0 tx=0).stop 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 msgr2=0x7febd41a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd41a1100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febcc02c180 tx=0x7febcc02c650 comp rx=0 tx=0).stop 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 shutdown_connections 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7febc00a2880 0x7febc00a4d40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd4105490 0x7febd41a1100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febd4104ad0 0x7febd419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 --2- 172.21.15.99:0/2003628255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd41038e0 0x7febd4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.016+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 >> 172.21.15.99:0/2003628255 conn(0x7febd40fee50 msgr2=0x7febd4107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:45.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.017+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 shutdown_connections 2024-03-23T23:43:45.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.017+0000 7febdc800640 1 -- 172.21.15.99:0/2003628255 wait complete. 2024-03-23T23:43:45.036 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.233+0000 7fc8f3600640 1 -- 172.21.15.99:0/2235351696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 msgr2=0x7fc8ec103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.233+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2235351696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec103d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 -- 172.21.15.99:0/2235351696 shutdown_connections 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2235351696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 0x7fc8ec10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2235351696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2235351696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec103d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 -- 172.21.15.99:0/2235351696 >> 172.21.15.99:0/2235351696 conn(0x7fc8ec0fee50 msgr2=0x7fc8ec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 -- 172.21.15.99:0/2235351696 shutdown_connections 2024-03-23T23:43:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.234+0000 7fc8f3600640 1 -- 172.21.15.99:0/2235351696 wait complete. 2024-03-23T23:43:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.235+0000 7fc8f3600640 1 Processor -- start 2024-03-23T23:43:45.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.235+0000 7fc8f3600640 1 -- start start 2024-03-23T23:43:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 0x7fc8ec1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec10f580 con 0x7fc8ec1038e0 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec10f400 con 0x7fc8ec105490 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f3600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec10f700 con 0x7fc8ec104ad0 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55444/0 (socket says 172.21.15.99:55444) 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f0c00640 1 -- 172.21.15.99:0/2450738705 learned_addr learned my addr 172.21.15.99:0/2450738705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.236+0000 7fc8f1600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 0x7fc8ec1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 -- 172.21.15.99:0/2450738705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 msgr2=0x7fc8ec19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 -- 172.21.15.99:0/2450738705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 msgr2=0x7fc8ec1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 0x7fc8ec1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 -- 172.21.15.99:0/2450738705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec1a1880 con 0x7fc8ec1038e0 2024-03-23T23:43:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8ebe00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f0c00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0043020 con 0x7fc8ec1038e0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.237+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec1a1b10 con 0x7fc8ec1038e0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.238+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec1a20b0 con 0x7fc8ec1038e0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.238+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8e00054e0 con 0x7fc8ec1038e0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.238+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0003bd0 con 0x7fc8ec1038e0 2024-03-23T23:43:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.239+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec10ac20 con 0x7fc8ec1038e0 2024-03-23T23:43:45.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:44 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2856546565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:45.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:44 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:44 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3633003701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:45.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.241+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc8e0046020 con 0x7fc8ec1038e0 2024-03-23T23:43:45.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.243+0000 7fc8e9600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 0x7fc8d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.244+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc8e00f0ae0 con 0x7fc8ec1038e0 2024-03-23T23:43:45.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.244+0000 7fc8ebe00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 0x7fc8d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.245+0000 7fc8ebe00640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 0x7fc8d40a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009fc0 tx=0x7fc8d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:45.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.250+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8e004d020 con 0x7fc8ec1038e0 2024-03-23T23:43:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:44 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2856546565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:44 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:44 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3633003701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:45.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:44 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2856546565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:45.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:44 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:45.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:44 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3633003701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:45.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.556+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8ec0622a0 con 0x7fc8ec1038e0 2024-03-23T23:43:45.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.557+0000 7fc8e9600640 1 -- 172.21.15.99:0/2450738705 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8e00bdb00 con 0x7fc8ec1038e0 2024-03-23T23:43:45.557 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:43:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 msgr2=0x7fc8d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 0x7fc8d40a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009fc0 tx=0x7fc8d8005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 msgr2=0x7fc8ec199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e002c650 comp rx=0 tx=0).stop 2024-03-23T23:43:45.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 shutdown_connections 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc8d40a2880 0x7fc8d40a4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec105490 0x7fc8ec1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec104ad0 0x7fc8ec19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 --2- 172.21.15.99:0/2450738705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8ec1038e0 0x7fc8ec199c70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 >> 172.21.15.99:0/2450738705 conn(0x7fc8ec0fee50 msgr2=0x7fc8ec107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 shutdown_connections 2024-03-23T23:43:45.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.559+0000 7fc8f3600640 1 -- 172.21.15.99:0/2450738705 wait complete. 2024-03-23T23:43:45.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.783+0000 7fe010e00640 1 -- 172.21.15.99:0/1130105368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 msgr2=0x7fe00c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.783+0000 7fe010e00640 1 --2- 172.21.15.99:0/1130105368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c103d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009860 tx=0x7fdffc02bb70 comp rx=0 tx=0).stop 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 -- 172.21.15.99:0/1130105368 shutdown_connections 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 --2- 172.21.15.99:0/1130105368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c105490 0x7fe00c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 --2- 172.21.15.99:0/1130105368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c104ad0 0x7fe00c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 --2- 172.21.15.99:0/1130105368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c103d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 -- 172.21.15.99:0/1130105368 >> 172.21.15.99:0/1130105368 conn(0x7fe00c0fee50 msgr2=0x7fe00c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:45.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 -- 172.21.15.99:0/1130105368 shutdown_connections 2024-03-23T23:43:45.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.784+0000 7fe010e00640 1 -- 172.21.15.99:0/1130105368 wait complete. 2024-03-23T23:43:45.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.785+0000 7fe010e00640 1 Processor -- start 2024-03-23T23:43:45.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.785+0000 7fe010e00640 1 -- start start 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 0x7fe00c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c105490 0x7fe00c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c10f530 con 0x7fe00c1038e0 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c10f3b0 con 0x7fe00c105490 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe010e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c10f6b0 con 0x7fe00c104ad0 2024-03-23T23:43:45.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 0x7fe00c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55454/0 (socket says 172.21.15.99:55454) 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009e00640 1 -- 172.21.15.99:0/4035352437 learned_addr learned my addr 172.21.15.99:0/4035352437 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe00a800640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c105490 0x7fe00c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 -- 172.21.15.99:0/4035352437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c105490 msgr2=0x7fe00c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c105490 0x7fe00c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 -- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 msgr2=0x7fe00c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:45.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.786+0000 7fe009400640 1 -- 172.21.15.99:0/4035352437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c1a1840 con 0x7fe00c104ad0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe009400640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 0x7fe00c19a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800cd00 con 0x7fe00c104ad0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c1a1b30 con 0x7fe00c104ad0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdff80035f0 con 0x7fe00c104ad0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8010690 con 0x7fe00c104ad0 2024-03-23T23:43:45.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.787+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c1a20d0 con 0x7fe00c104ad0 2024-03-23T23:43:45.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.789+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c10ac80 con 0x7fe00c104ad0 2024-03-23T23:43:45.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.791+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdff80037a0 con 0x7fe00c104ad0 2024-03-23T23:43:45.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.793+0000 7fe002a00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 0x7fdff00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:45.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.793+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdff80ced00 con 0x7fe00c104ad0 2024-03-23T23:43:45.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.793+0000 7fe009e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 0x7fdff00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:45.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.794+0000 7fe009e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 0x7fdff00a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:45.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:45.796+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdff8009510 con 0x7fe00c104ad0 2024-03-23T23:43:46.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.071+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe00c0622a0 con 0x7fe00c104ad0 2024-03-23T23:43:46.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.072+0000 7fe002a00640 1 -- 172.21.15.99:0/4035352437 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdff809bd80 con 0x7fe00c104ad0 2024-03-23T23:43:46.072 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 msgr2=0x7fdff00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 0x7fdff00a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc034ab0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 msgr2=0x7fe00c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 0x7fe00c19a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff800ee50 comp rx=0 tx=0).stop 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 shutdown_connections 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdff00a2800 0x7fdff00a4cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c105490 0x7fe00c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c104ad0 0x7fe00c19a170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 --2- 172.21.15.99:0/4035352437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe00c1038e0 0x7fe00c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 >> 172.21.15.99:0/4035352437 conn(0x7fe00c0fee50 msgr2=0x7fe00c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:46.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 shutdown_connections 2024-03-23T23:43:46.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.074+0000 7fe010e00640 1 -- 172.21.15.99:0/4035352437 wait complete. 2024-03-23T23:43:46.092 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-23T23:43:46.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:43:46.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 -- 172.21.15.99:0/1750492196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0050103310 msgr2=0x7f0050109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 --2- 172.21.15.99:0/1750492196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0050103310 0x7f0050109ba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0048009cd0 tx=0x7f004802c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 -- 172.21.15.99:0/1750492196 shutdown_connections 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 --2- 172.21.15.99:0/1750492196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0050103310 0x7f0050109ba0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 --2- 172.21.15.99:0/1750492196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 --2- 172.21.15.99:0/1750492196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00500ff980 0x7f00500ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 -- 172.21.15.99:0/1750492196 >> 172.21.15.99:0/1750492196 conn(0x7f00500fb2d0 msgr2=0x7f00500fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 -- 172.21.15.99:0/1750492196 shutdown_connections 2024-03-23T23:43:46.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.286+0000 7f0057c00640 1 -- 172.21.15.99:0/1750492196 wait complete. 2024-03-23T23:43:46.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.287+0000 7f0057c00640 1 Processor -- start 2024-03-23T23:43:46.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.287+0000 7f0057c00640 1 -- start start 2024-03-23T23:43:46.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 0x7f0050197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050103310 0x7f005019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005010d330 con 0x7f0050103310 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005010d1b0 con 0x7f0050102950 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0057c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005010d4b0 con 0x7f00500ff980 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0055200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 0x7f0050197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0055200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 0x7f0050197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57458/0 (socket says 172.21.15.99:57458) 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0055200640 1 -- 172.21.15.99:0/1966536961 learned_addr learned my addr 172.21.15.99:0/1966536961 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42384/0 (socket says 172.21.15.99:42384) 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f0055c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050103310 0x7f005019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 -- 172.21.15.99:0/1966536961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 msgr2=0x7f0050197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 0x7f0050197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 -- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050103310 msgr2=0x7f005019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050103310 0x7f005019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.288+0000 7f004fe00640 1 -- 172.21.15.99:0/1966536961 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005019f590 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f004fe00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e9f0 tx=0x7f003c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00ccc0 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f003c010910 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c018960 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005019f880 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.289+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005019fe20 con 0x7f0050102950 2024-03-23T23:43:46.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.290+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0050108a80 con 0x7f0050102950 2024-03-23T23:43:46.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.293+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f003c010ac0 con 0x7f0050102950 2024-03-23T23:43:46.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.294+0000 7f004d600640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 0x7f00380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.294+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f003c0cee10 con 0x7f0050102950 2024-03-23T23:43:46.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.294+0000 7f0055200640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 0x7f00380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.295+0000 7f0055200640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 0x7f00380a4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00440058c0 tx=0x7f0044005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:46.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.301+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f003c0d2030 con 0x7f0050102950 2024-03-23T23:43:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:45 smithi099 ceph-mon[31715]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:43:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:45 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2003628255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:45 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:45 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2450738705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:45 smithi149 ceph-mon[36083]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:43:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:45 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2003628255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:45 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:45 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2450738705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:45 smithi174 ceph-mon[37002]: pgmap v396: 97 pgs: 97 active+clean; 646 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:43:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:45 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2003628255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:45 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:45 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2450738705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.618+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00500622a0 con 0x7f0050102950 2024-03-23T23:43:46.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.619+0000 7f004d600640 1 -- 172.21.15.99:0/1966536961 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f003c09b7e0 con 0x7f0050102950 2024-03-23T23:43:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:46.620 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:43:46.620 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:43:46.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.622+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 msgr2=0x7f00380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.622+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 0x7f00380a4d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00440058c0 tx=0x7f0044005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.622+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 msgr2=0x7f0050197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.622+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e9f0 tx=0x7f003c00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 shutdown_connections 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00380a2880 0x7f00380a4d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050103310 0x7f005019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0050102950 0x7f0050197eb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 --2- 172.21.15.99:0/1966536961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00500ff980 0x7f0050197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 >> 172.21.15.99:0/1966536961 conn(0x7f00500fb2d0 msgr2=0x7f0050105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:46.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 shutdown_connections 2024-03-23T23:43:46.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.623+0000 7f0057c00640 1 -- 172.21.15.99:0/1966536961 wait complete. 2024-03-23T23:43:46.641 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-23T23:43:46.641 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-23T23:43:46.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T23:43:46.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 -- 172.21.15.99:0/561139832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 msgr2=0x7f5530073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 --2- 172.21.15.99:0/561139832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530073ad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002bcd0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 -- 172.21.15.99:0/561139832 shutdown_connections 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 --2- 172.21.15.99:0/561139832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 0x7f553010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 --2- 172.21.15.99:0/561139832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 --2- 172.21.15.99:0/561139832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530073ad0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 -- 172.21.15.99:0/561139832 >> 172.21.15.99:0/561139832 conn(0x7f55300fca70 msgr2=0x7f55300feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 -- 172.21.15.99:0/561139832 shutdown_connections 2024-03-23T23:43:46.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.853+0000 7f5534a00640 1 -- 172.21.15.99:0/561139832 wait complete. 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.854+0000 7f5534a00640 1 Processor -- start 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 -- start start 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 0x7f55301a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553010f3c0 con 0x7f5530109390 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553010f240 con 0x7f55300740a0 2024-03-23T23:43:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f5534a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553010f540 con 0x7f5530075680 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:42396/0 (socket says 172.21.15.99:42396) 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57480/0 (socket says 172.21.15.99:57480) 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552f400640 1 -- 172.21.15.99:0/3368661093 learned_addr learned my addr 172.21.15.99:0/3368661093 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.855+0000 7f552fe00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 0x7f55301a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 -- 172.21.15.99:0/3368661093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 msgr2=0x7f5530199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 -- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 msgr2=0x7f55301a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:46.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 0x7f55301a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 -- 172.21.15.99:0/3368661093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55301a1600 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552ea00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f552f400640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5520000c00 tx=0x7f552002c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520043020 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.856+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55301a1890 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.857+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55200054e0 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.857+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520003c20 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.857+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55301a1e30 con 0x7f55300740a0 2024-03-23T23:43:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.858+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55300683b0 con 0x7f55300740a0 2024-03-23T23:43:46.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.861+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5520046020 con 0x7f55300740a0 2024-03-23T23:43:46.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.862+0000 7f5527e00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 0x7f55140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:46.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.862+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55200f0a50 con 0x7f55300740a0 2024-03-23T23:43:46.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.862+0000 7f552ea00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 0x7f55140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:46.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.863+0000 7f552ea00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 0x7f55140a4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f553019aef0 tx=0x7f551c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:46.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:46.866+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f552004d020 con 0x7f55300740a0 2024-03-23T23:43:46.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:46 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/4035352437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:46.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:46.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:46 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1966536961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:46.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:46 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:47.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.066+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5530061ba0 con 0x7f55140a2880 2024-03-23T23:43:47.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.074+0000 7f5527e00640 1 -- 172.21.15.99:0/3368661093 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259193 (secure 0 0 0) 0x7f5530061ba0 con 0x7f55140a2880 2024-03-23T23:43:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:47.080 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:43:47.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 msgr2=0x7f55140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 0x7f55140a4d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f553019aef0 tx=0x7f551c005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 msgr2=0x7f5530199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5520000c00 tx=0x7f552002c6f0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 shutdown_connections 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f55140a2880 0x7f55140a4d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530109390 0x7f55301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5530075680 0x7f5530199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 --2- 172.21.15.99:0/3368661093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55300740a0 0x7f5530199a30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 >> 172.21.15.99:0/3368661093 conn(0x7f55300fca70 msgr2=0x7f5530105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 shutdown_connections 2024-03-23T23:43:47.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.080+0000 7f5534a00640 1 -- 172.21.15.99:0/3368661093 wait complete. 2024-03-23T23:43:47.099 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-03-23T23:43:46.780875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666873,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1637,"num_read_kb":949,"num_write":438,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381676,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560148,"statfs":{"total":1151940427776,"available":1151549591552,"internally_reserved":0,"allocated":12005376,"data_stored":7470279,"data_compressed":181782,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":73,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003960"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751543+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:42:36.751543+0000","last_peered":"2024-03-23T23:42:36.751543+0000","last_clean":"2024-03-23T23:42:36.751543+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:42:36.751543+0000","last_undegraded":"2024-03-23T23:42:36.751543+0000","last_fullsized":"2024-03-23T23:42:36.751543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758825+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:42:36.758825+0000","last_peered":"2024-03-23T23:42:36.758825+0000","last_clean":"2024-03-23T23:42:36.758825+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:42:36.758825+0000","last_undegraded":"2024-03-23T23:42:36.758825+0000","last_fullsized":"2024-03-23T23:42:36.758825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761403+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:42:36.761403+0000","last_peered":"2024-03-23T23:42:36.761403+0000","last_clean":"2024-03-23T23:42:36.761403+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:42:36.761403+0000","last_undegraded":"2024-03-23T23:42:36.761403+0000","last_fullsized":"2024-03-23T23:42:36.761403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.723120+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:42:36.723120+0000","last_peered":"2024-03-23T23:42:36.723120+0000","last_clean":"2024-03-23T23:42:36.723120+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:42:36.723120+0000","last_undegraded":"2024-03-23T23:42:36.723120+0000","last_fullsized":"2024-03-23T23:42:36.723120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725071+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:42:36.725071+0000","last_peered":"2024-03-23T23:42:36.725071+0000","last_clean":"2024-03-23T23:42:36.725071+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:42:36.725071+0000","last_undegraded":"2024-03-23T23:42:36.725071+0000","last_fullsized":"2024-03-23T23:42:36.725071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724996+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:42:36.724996+0000","last_peered":"2024-03-23T23:42:36.724996+0000","last_clean":"2024-03-23T23:42:36.724996+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:42:36.724996+0000","last_undegraded":"2024-03-23T23:42:36.724996+0000","last_fullsized":"2024-03-23T23:42:36.724996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748285+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:42:36.748285+0000","last_peered":"2024-03-23T23:42:36.748285+0000","last_clean":"2024-03-23T23:42:36.748285+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:42:36.748285+0000","last_undegraded":"2024-03-23T23:42:36.748285+0000","last_fullsized":"2024-03-23T23:42:36.748285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738694+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:42:36.738694+0000","last_peered":"2024-03-23T23:42:36.738694+0000","last_clean":"2024-03-23T23:42:36.738694+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:42:36.738694+0000","last_undegraded":"2024-03-23T23:42:36.738694+0000","last_fullsized":"2024-03-23T23:42:36.738694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728477+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:42:36.728477+0000","last_peered":"2024-03-23T23:42:36.728477+0000","last_clean":"2024-03-23T23:42:36.728477+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:42:36.728477+0000","last_undegraded":"2024-03-23T23:42:36.728477+0000","last_fullsized":"2024-03-23T23:42:36.728477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724950+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:42:36.724950+0000","last_peered":"2024-03-23T23:42:36.724950+0000","last_clean":"2024-03-23T23:42:36.724950+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:42:36.724950+0000","last_undegraded":"2024-03-23T23:42:36.724950+0000","last_fullsized":"2024-03-23T23:42:36.724950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.720950+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:42:36.720950+0000","last_peered":"2024-03-23T23:42:36.720950+0000","last_clean":"2024-03-23T23:42:36.720950+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:42:36.720950+0000","last_undegraded":"2024-03-23T23:42:36.720950+0000","last_fullsized":"2024-03-23T23:42:36.720950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.715575+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:42:36.715575+0000","last_peered":"2024-03-23T23:42:36.715575+0000","last_clean":"2024-03-23T23:42:36.715575+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:42:36.715575+0000","last_undegraded":"2024-03-23T23:42:36.715575+0000","last_fullsized":"2024-03-23T23:42:36.715575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.746803+0000","last_change":"2024-03-23T23:42:07.892099+0000","last_active":"2024-03-23T23:42:36.746803+0000","last_peered":"2024-03-23T23:42:36.746803+0000","last_clean":"2024-03-23T23:42:36.746803+0000","last_became_active":"2024-03-23T23:42:07.891702+0000","last_became_peered":"2024-03-23T23:42:07.891702+0000","last_unstale":"2024-03-23T23:42:36.746803+0000","last_undegraded":"2024-03-23T23:42:36.746803+0000","last_fullsized":"2024-03-23T23:42:36.746803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:52:44.779482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760590+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:42:36.760590+0000","last_peered":"2024-03-23T23:42:36.760590+0000","last_clean":"2024-03-23T23:42:36.760590+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:42:36.760590+0000","last_undegraded":"2024-03-23T23:42:36.760590+0000","last_fullsized":"2024-03-23T23:42:36.760590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758705+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:42:36.758705+0000","last_peered":"2024-03-23T23:42:36.758705+0000","last_clean":"2024-03-23T23:42:36.758705+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:42:36.758705+0000","last_undegraded":"2024-03-23T23:42:36.758705+0000","last_fullsized":"2024-03-23T23:42:36.758705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758661+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:42:36.758661+0000","last_peered":"2024-03-23T23:42:36.758661+0000","last_clean":"2024-03-23T23:42:36.758661+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:42:36.758661+0000","last_undegraded":"2024-03-23T23:42:36.758661+0000","last_fullsized":"2024-03-23T23:42:36.758661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751658+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:42:36.751658+0000","last_peered":"2024-03-23T23:42:36.751658+0000","last_clean":"2024-03-23T23:42:36.751658+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:42:36.751658+0000","last_undegraded":"2024-03-23T23:42:36.751658+0000","last_fullsized":"2024-03-23T23:42:36.751658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749416+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:42:36.749416+0000","last_peered":"2024-03-23T23:42:36.749416+0000","last_clean":"2024-03-23T23:42:36.749416+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:42:36.749416+0000","last_undegraded":"2024-03-23T23:42:36.749416+0000","last_fullsized":"2024-03-23T23:42:36.749416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.721815+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:42:36.721815+0000","last_peered":"2024-03-23T23:42:36.721815+0000","last_clean":"2024-03-23T23:42:36.721815+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:42:36.721815+0000","last_undegraded":"2024-03-23T23:42:36.721815+0000","last_fullsized":"2024-03-23T23:42:36.721815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":180,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:41.480881+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:43:41.480881+0000","last_peered":"2024-03-23T23:43:41.480881+0000","last_clean":"2024-03-23T23:43:41.480881+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:43:41.480881+0000","last_undegraded":"2024-03-23T23:43:41.480881+0000","last_fullsized":"2024-03-23T23:43:41.480881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740593+0000","last_change":"2024-03-23T23:42:07.890419+0000","last_active":"2024-03-23T23:42:36.740593+0000","last_peered":"2024-03-23T23:42:36.740593+0000","last_clean":"2024-03-23T23:42:36.740593+0000","last_became_active":"2024-03-23T23:42:07.889998+0000","last_became_peered":"2024-03-23T23:42:07.889998+0000","last_unstale":"2024-03-23T23:42:36.740593+0000","last_undegraded":"2024-03-23T23:42:36.740593+0000","last_fullsized":"2024-03-23T23:42:36.740593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:35:13.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.672810+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:42:08.672810+0000","last_peered":"2024-03-23T23:42:08.672810+0000","last_clean":"2024-03-23T23:42:08.672810+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:42:08.672810+0000","last_undegraded":"2024-03-23T23:42:08.672810+0000","last_fullsized":"2024-03-23T23:42:08.672810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761538+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:42:36.761538+0000","last_peered":"2024-03-23T23:42:36.761538+0000","last_clean":"2024-03-23T23:42:36.761538+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:42:36.761538+0000","last_undegraded":"2024-03-23T23:42:36.761538+0000","last_fullsized":"2024-03-23T23:42:36.761538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.892818+0000","last_change":"2024-03-23T23:42:07.892818+0000","last_active":"2024-03-23T23:42:07.892818+0000","last_peered":"2024-03-23T23:42:07.892818+0000","last_clean":"2024-03-23T23:42:07.892818+0000","last_became_active":"2024-03-23T23:42:07.880207+0000","last_became_peered":"2024-03-23T23:42:07.880207+0000","last_unstale":"2024-03-23T23:42:07.892818+0000","last_undegraded":"2024-03-23T23:42:07.892818+0000","last_fullsized":"2024-03-23T23:42:07.892818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:04:54.088457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743660+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:42:36.743660+0000","last_peered":"2024-03-23T23:42:36.743660+0000","last_clean":"2024-03-23T23:42:36.743660+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:42:36.743660+0000","last_undegraded":"2024-03-23T23:42:36.743660+0000","last_fullsized":"2024-03-23T23:42:36.743660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876310+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:42:07.876310+0000","last_peered":"2024-03-23T23:42:07.876310+0000","last_clean":"2024-03-23T23:42:07.876310+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:42:07.876310+0000","last_undegraded":"2024-03-23T23:42:07.876310+0000","last_fullsized":"2024-03-23T23:42:07.876310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727449+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:42:36.727449+0000","last_peered":"2024-03-23T23:42:36.727449+0000","last_clean":"2024-03-23T23:42:36.727449+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:42:36.727449+0000","last_undegraded":"2024-03-23T23:42:36.727449+0000","last_fullsized":"2024-03-23T23:42:36.727449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":187,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.475530+0000","last_change":"2024-03-23T23:42:08.388477+0000","last_active":"2024-03-23T23:43:42.475530+0000","last_peered":"2024-03-23T23:43:42.475530+0000","last_clean":"2024-03-23T23:43:42.475530+0000","last_became_active":"2024-03-23T23:42:08.384694+0000","last_became_peered":"2024-03-23T23:42:08.384694+0000","last_unstale":"2024-03-23T23:43:42.475530+0000","last_undegraded":"2024-03-23T23:43:42.475530+0000","last_fullsized":"2024-03-23T23:43:42.475530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:13.778147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.718769+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.718769+0000","last_peered":"2024-03-23T23:42:36.718769+0000","last_clean":"2024-03-23T23:42:36.718769+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:42:36.718769+0000","last_undegraded":"2024-03-23T23:42:36.718769+0000","last_fullsized":"2024-03-23T23:42:36.718769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.882785+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:42:07.882785+0000","last_peered":"2024-03-23T23:42:07.882785+0000","last_clean":"2024-03-23T23:42:07.882785+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:42:07.882785+0000","last_undegraded":"2024-03-23T23:42:07.882785+0000","last_fullsized":"2024-03-23T23:42:07.882785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743549+0000","last_change":"2024-03-23T23:42:08.673791+0000","last_active":"2024-03-23T23:42:36.743549+0000","last_peered":"2024-03-23T23:42:36.743549+0000","last_clean":"2024-03-23T23:42:36.743549+0000","last_became_active":"2024-03-23T23:42:08.673494+0000","last_became_peered":"2024-03-23T23:42:08.673494+0000","last_unstale":"2024-03-23T23:42:36.743549+0000","last_undegraded":"2024-03-23T23:42:36.743549+0000","last_fullsized":"2024-03-23T23:42:36.743549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:22:10.209266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.877587+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:42:07.877587+0000","last_peered":"2024-03-23T23:42:07.877587+0000","last_clean":"2024-03-23T23:42:07.877587+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:42:07.877587+0000","last_undegraded":"2024-03-23T23:42:07.877587+0000","last_fullsized":"2024-03-23T23:42:07.877587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738549+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:42:36.738549+0000","last_peered":"2024-03-23T23:42:36.738549+0000","last_clean":"2024-03-23T23:42:36.738549+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:42:36.738549+0000","last_undegraded":"2024-03-23T23:42:36.738549+0000","last_fullsized":"2024-03-23T23:42:36.738549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.880640+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:42:07.880640+0000","last_peered":"2024-03-23T23:42:07.880640+0000","last_clean":"2024-03-23T23:42:07.880640+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:42:07.880640+0000","last_undegraded":"2024-03-23T23:42:07.880640+0000","last_fullsized":"2024-03-23T23:42:07.880640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.750029+0000","last_change":"2024-03-23T23:42:07.891565+0000","last_active":"2024-03-23T23:42:36.750029+0000","last_peered":"2024-03-23T23:42:36.750029+0000","last_clean":"2024-03-23T23:42:36.750029+0000","last_became_active":"2024-03-23T23:42:07.890328+0000","last_became_peered":"2024-03-23T23:42:07.890328+0000","last_unstale":"2024-03-23T23:42:36.750029+0000","last_undegraded":"2024-03-23T23:42:36.750029+0000","last_fullsized":"2024-03-23T23:42:36.750029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:44:07.467469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":322,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.444327+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:43:45.444327+0000","last_peered":"2024-03-23T23:43:45.444327+0000","last_clean":"2024-03-23T23:43:45.444327+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:43:45.444327+0000","last_undegraded":"2024-03-23T23:43:45.444327+0000","last_fullsized":"2024-03-23T23:43:45.444327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742475+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:42:36.742475+0000","last_peered":"2024-03-23T23:42:36.742475+0000","last_clean":"2024-03-23T23:42:36.742475+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:42:36.742475+0000","last_undegraded":"2024-03-23T23:42:36.742475+0000","last_fullsized":"2024-03-23T23:42:36.742475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878129+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:42:07.878129+0000","last_peered":"2024-03-23T23:42:07.878129+0000","last_clean":"2024-03-23T23:42:07.878129+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:42:07.878129+0000","last_undegraded":"2024-03-23T23:42:07.878129+0000","last_fullsized":"2024-03-23T23:42:07.878129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760824+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:42:36.760824+0000","last_peered":"2024-03-23T23:42:36.760824+0000","last_clean":"2024-03-23T23:42:36.760824+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:42:36.760824+0000","last_undegraded":"2024-03-23T23:42:36.760824+0000","last_fullsized":"2024-03-23T23:42:36.760824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881851+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:42:07.881851+0000","last_peered":"2024-03-23T23:42:07.881851+0000","last_clean":"2024-03-23T23:42:07.881851+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:42:07.881851+0000","last_undegraded":"2024-03-23T23:42:07.881851+0000","last_fullsized":"2024-03-23T23:42:07.881851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728538+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:42:36.728538+0000","last_peered":"2024-03-23T23:42:36.728538+0000","last_clean":"2024-03-23T23:42:36.728538+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:42:36.728538+0000","last_undegraded":"2024-03-23T23:42:36.728538+0000","last_fullsized":"2024-03-23T23:42:36.728538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":366,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.472694+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:43:45.472694+0000","last_peered":"2024-03-23T23:43:45.472694+0000","last_clean":"2024-03-23T23:43:45.472694+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:43:45.472694+0000","last_undegraded":"2024-03-23T23:43:45.472694+0000","last_fullsized":"2024-03-23T23:43:45.472694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":307,"num_read_kb":221,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726491+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:42:36.726491+0000","last_peered":"2024-03-23T23:42:36.726491+0000","last_clean":"2024-03-23T23:42:36.726491+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:42:36.726491+0000","last_undegraded":"2024-03-23T23:42:36.726491+0000","last_fullsized":"2024-03-23T23:42:36.726491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878343+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:42:07.878343+0000","last_peered":"2024-03-23T23:42:07.878343+0000","last_clean":"2024-03-23T23:42:07.878343+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:42:07.878343+0000","last_undegraded":"2024-03-23T23:42:07.878343+0000","last_fullsized":"2024-03-23T23:42:07.878343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740369+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:42:36.740369+0000","last_peered":"2024-03-23T23:42:36.740369+0000","last_clean":"2024-03-23T23:42:36.740369+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:42:36.740369+0000","last_undegraded":"2024-03-23T23:42:36.740369+0000","last_fullsized":"2024-03-23T23:42:36.740369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881121+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:42:07.881121+0000","last_peered":"2024-03-23T23:42:07.881121+0000","last_clean":"2024-03-23T23:42:07.881121+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:42:07.881121+0000","last_undegraded":"2024-03-23T23:42:07.881121+0000","last_fullsized":"2024-03-23T23:42:07.881121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.737057+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:42:36.737057+0000","last_peered":"2024-03-23T23:42:36.737057+0000","last_clean":"2024-03-23T23:42:36.737057+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:42:36.737057+0000","last_undegraded":"2024-03-23T23:42:36.737057+0000","last_fullsized":"2024-03-23T23:42:36.737057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.497351+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:43:42.497351+0000","last_peered":"2024-03-23T23:43:42.497351+0000","last_clean":"2024-03-23T23:43:42.497351+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:43:42.497351+0000","last_undegraded":"2024-03-23T23:43:42.497351+0000","last_fullsized":"2024-03-23T23:43:42.497351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1426,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725438+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:42:36.725438+0000","last_peered":"2024-03-23T23:42:36.725438+0000","last_clean":"2024-03-23T23:42:36.725438+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:42:36.725438+0000","last_undegraded":"2024-03-23T23:42:36.725438+0000","last_fullsized":"2024-03-23T23:42:36.725438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876027+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:42:07.876027+0000","last_peered":"2024-03-23T23:42:07.876027+0000","last_clean":"2024-03-23T23:42:07.876027+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:42:07.876027+0000","last_undegraded":"2024-03-23T23:42:07.876027+0000","last_fullsized":"2024-03-23T23:42:07.876027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726840+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:42:36.726840+0000","last_peered":"2024-03-23T23:42:36.726840+0000","last_clean":"2024-03-23T23:42:36.726840+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:42:36.726840+0000","last_undegraded":"2024-03-23T23:42:36.726840+0000","last_fullsized":"2024-03-23T23:42:36.726840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876507+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:42:07.876507+0000","last_peered":"2024-03-23T23:42:07.876507+0000","last_clean":"2024-03-23T23:42:07.876507+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:42:07.876507+0000","last_undegraded":"2024-03-23T23:42:07.876507+0000","last_fullsized":"2024-03-23T23:42:07.876507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722599+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:42:36.722599+0000","last_peered":"2024-03-23T23:42:36.722599+0000","last_clean":"2024-03-23T23:42:36.722599+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:42:36.722599+0000","last_undegraded":"2024-03-23T23:42:36.722599+0000","last_fullsized":"2024-03-23T23:42:36.722599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.888239+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:42:07.888239+0000","last_peered":"2024-03-23T23:42:07.888239+0000","last_clean":"2024-03-23T23:42:07.888239+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:42:07.888239+0000","last_undegraded":"2024-03-23T23:42:07.888239+0000","last_fullsized":"2024-03-23T23:42:07.888239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728445+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:42:36.728445+0000","last_peered":"2024-03-23T23:42:36.728445+0000","last_clean":"2024-03-23T23:42:36.728445+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:42:36.728445+0000","last_undegraded":"2024-03-23T23:42:36.728445+0000","last_fullsized":"2024-03-23T23:42:36.728445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:43.484227+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:43:43.484227+0000","last_peered":"2024-03-23T23:43:43.484227+0000","last_clean":"2024-03-23T23:43:43.484227+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:43:43.484227+0000","last_undegraded":"2024-03-23T23:43:43.484227+0000","last_fullsized":"2024-03-23T23:43:43.484227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.757784+0000","last_change":"2024-03-23T23:42:07.890902+0000","last_active":"2024-03-23T23:42:36.757784+0000","last_peered":"2024-03-23T23:42:36.757784+0000","last_clean":"2024-03-23T23:42:36.757784+0000","last_became_active":"2024-03-23T23:42:07.890514+0000","last_became_peered":"2024-03-23T23:42:07.890514+0000","last_unstale":"2024-03-23T23:42:36.757784+0000","last_undegraded":"2024-03-23T23:42:36.757784+0000","last_fullsized":"2024-03-23T23:42:36.757784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:42.343732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726405+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:42:36.726405+0000","last_peered":"2024-03-23T23:42:36.726405+0000","last_clean":"2024-03-23T23:42:36.726405+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:42:36.726405+0000","last_undegraded":"2024-03-23T23:42:36.726405+0000","last_fullsized":"2024-03-23T23:42:36.726405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'7","reported_seq":173,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:40.854661+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:43:40.854661+0000","last_peered":"2024-03-23T23:43:40.854661+0000","last_clean":"2024-03-23T23:43:40.854661+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:43:40.854661+0000","last_undegraded":"2024-03-23T23:43:40.854661+0000","last_fullsized":"2024-03-23T23:43:40.854661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729867+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:42:36.729867+0000","last_peered":"2024-03-23T23:42:36.729867+0000","last_clean":"2024-03-23T23:42:36.729867+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:42:36.729867+0000","last_undegraded":"2024-03-23T23:42:36.729867+0000","last_fullsized":"2024-03-23T23:42:36.729867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360595+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:42:08.360595+0000","last_peered":"2024-03-23T23:42:08.360595+0000","last_clean":"2024-03-23T23:42:08.360595+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:42:08.360595+0000","last_undegraded":"2024-03-23T23:42:08.360595+0000","last_fullsized":"2024-03-23T23:42:08.360595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878026+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:42:07.878026+0000","last_peered":"2024-03-23T23:42:07.878026+0000","last_clean":"2024-03-23T23:42:07.878026+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:42:07.878026+0000","last_undegraded":"2024-03-23T23:42:07.878026+0000","last_fullsized":"2024-03-23T23:42:07.878026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730425+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:42:36.730425+0000","last_peered":"2024-03-23T23:42:36.730425+0000","last_clean":"2024-03-23T23:42:36.730425+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:42:36.730425+0000","last_undegraded":"2024-03-23T23:42:36.730425+0000","last_fullsized":"2024-03-23T23:42:36.730425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.895117+0000","last_change":"2024-03-23T23:42:07.895117+0000","last_active":"2024-03-23T23:42:07.895117+0000","last_peered":"2024-03-23T23:42:07.895117+0000","last_clean":"2024-03-23T23:42:07.895117+0000","last_became_active":"2024-03-23T23:42:07.881460+0000","last_became_peered":"2024-03-23T23:42:07.881460+0000","last_unstale":"2024-03-23T23:42:07.895117+0000","last_undegraded":"2024-03-23T23:42:07.895117+0000","last_fullsized":"2024-03-23T23:42:07.895117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:07.663519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876304+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:42:07.876304+0000","last_peered":"2024-03-23T23:42:07.876304+0000","last_clean":"2024-03-23T23:42:07.876304+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:42:07.876304+0000","last_undegraded":"2024-03-23T23:42:07.876304+0000","last_fullsized":"2024-03-23T23:42:07.876304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.744135+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:42:36.744135+0000","last_peered":"2024-03-23T23:42:36.744135+0000","last_clean":"2024-03-23T23:42:36.744135+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:42:36.744135+0000","last_undegraded":"2024-03-23T23:42:36.744135+0000","last_fullsized":"2024-03-23T23:42:36.744135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:18.409637+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:43:18.409637+0000","last_peered":"2024-03-23T23:43:18.409637+0000","last_clean":"2024-03-23T23:43:18.409637+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:43:18.409637+0000","last_undegraded":"2024-03-23T23:43:18.409637+0000","last_fullsized":"2024-03-23T23:43:18.409637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724617+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:42:36.724617+0000","last_peered":"2024-03-23T23:42:36.724617+0000","last_clean":"2024-03-23T23:42:36.724617+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:42:36.724617+0000","last_undegraded":"2024-03-23T23:42:36.724617+0000","last_fullsized":"2024-03-23T23:42:36.724617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360632+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:42:08.360632+0000","last_peered":"2024-03-23T23:42:08.360632+0000","last_clean":"2024-03-23T23:42:08.360632+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:42:08.360632+0000","last_undegraded":"2024-03-23T23:42:08.360632+0000","last_fullsized":"2024-03-23T23:42:08.360632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.759993+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:42:36.759993+0000","last_peered":"2024-03-23T23:42:36.759993+0000","last_clean":"2024-03-23T23:42:36.759993+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:42:36.759993+0000","last_undegraded":"2024-03-23T23:42:36.759993+0000","last_fullsized":"2024-03-23T23:42:36.759993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.383624+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:42:08.383624+0000","last_peered":"2024-03-23T23:42:08.383624+0000","last_clean":"2024-03-23T23:42:08.383624+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:42:08.383624+0000","last_undegraded":"2024-03-23T23:42:08.383624+0000","last_fullsized":"2024-03-23T23:42:08.383624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.704210+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:42:36.704210+0000","last_peered":"2024-03-23T23:42:36.704210+0000","last_clean":"2024-03-23T23:42:36.704210+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:42:36.704210+0000","last_undegraded":"2024-03-23T23:42:36.704210+0000","last_fullsized":"2024-03-23T23:42:36.704210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.671899+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:42:08.671899+0000","last_peered":"2024-03-23T23:42:08.671899+0000","last_clean":"2024-03-23T23:42:08.671899+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:42:08.671899+0000","last_undegraded":"2024-03-23T23:42:08.671899+0000","last_fullsized":"2024-03-23T23:42:08.671899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751177+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:42:36.751177+0000","last_peered":"2024-03-23T23:42:36.751177+0000","last_clean":"2024-03-23T23:42:36.751177+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:42:36.751177+0000","last_undegraded":"2024-03-23T23:42:36.751177+0000","last_fullsized":"2024-03-23T23:42:36.751177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878466+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:42:07.878466+0000","last_peered":"2024-03-23T23:42:07.878466+0000","last_clean":"2024-03-23T23:42:07.878466+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:42:07.878466+0000","last_undegraded":"2024-03-23T23:42:07.878466+0000","last_fullsized":"2024-03-23T23:42:07.878466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.755005+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:42:36.755005+0000","last_peered":"2024-03-23T23:42:36.755005+0000","last_clean":"2024-03-23T23:42:36.755005+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:42:36.755005+0000","last_undegraded":"2024-03-23T23:42:36.755005+0000","last_fullsized":"2024-03-23T23:42:36.755005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.365111+0000","last_change":"2024-03-23T23:42:08.364753+0000","last_active":"2024-03-23T23:42:08.365111+0000","last_peered":"2024-03-23T23:42:08.365111+0000","last_clean":"2024-03-23T23:42:08.365111+0000","last_became_active":"2024-03-23T23:42:08.361512+0000","last_became_peered":"2024-03-23T23:42:08.361512+0000","last_unstale":"2024-03-23T23:42:08.365111+0000","last_undegraded":"2024-03-23T23:42:08.365111+0000","last_fullsized":"2024-03-23T23:42:08.365111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:06:40.047574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.736240+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:42:36.736240+0000","last_peered":"2024-03-23T23:42:36.736240+0000","last_clean":"2024-03-23T23:42:36.736240+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:42:36.736240+0000","last_undegraded":"2024-03-23T23:42:36.736240+0000","last_fullsized":"2024-03-23T23:42:36.736240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.441432+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:43:42.441432+0000","last_peered":"2024-03-23T23:43:42.441432+0000","last_clean":"2024-03-23T23:43:42.441432+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:43:42.441432+0000","last_undegraded":"2024-03-23T23:43:42.441432+0000","last_fullsized":"2024-03-23T23:43:42.441432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727132+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:42:36.727132+0000","last_peered":"2024-03-23T23:42:36.727132+0000","last_clean":"2024-03-23T23:42:36.727132+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:42:36.727132+0000","last_undegraded":"2024-03-23T23:42:36.727132+0000","last_fullsized":"2024-03-23T23:42:36.727132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881681+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:42:07.881681+0000","last_peered":"2024-03-23T23:42:07.881681+0000","last_clean":"2024-03-23T23:42:07.881681+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:42:07.881681+0000","last_undegraded":"2024-03-23T23:42:07.881681+0000","last_fullsized":"2024-03-23T23:42:07.881681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725297+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:42:36.725297+0000","last_peered":"2024-03-23T23:42:36.725297+0000","last_clean":"2024-03-23T23:42:36.725297+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:42:36.725297+0000","last_undegraded":"2024-03-23T23:42:36.725297+0000","last_fullsized":"2024-03-23T23:42:36.725297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878327+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:42:07.878327+0000","last_peered":"2024-03-23T23:42:07.878327+0000","last_clean":"2024-03-23T23:42:07.878327+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:42:07.878327+0000","last_undegraded":"2024-03-23T23:42:07.878327+0000","last_fullsized":"2024-03-23T23:42:07.878327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727463+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:42:36.727463+0000","last_peered":"2024-03-23T23:42:36.727463+0000","last_clean":"2024-03-23T23:42:36.727463+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:42:36.727463+0000","last_undegraded":"2024-03-23T23:42:36.727463+0000","last_fullsized":"2024-03-23T23:42:36.727463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727456+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.727456+0000","last_peered":"2024-03-23T23:42:36.727456+0000","last_clean":"2024-03-23T23:42:36.727456+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:42:36.727456+0000","last_undegraded":"2024-03-23T23:42:36.727456+0000","last_fullsized":"2024-03-23T23:42:36.727456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724622+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:42:36.724622+0000","last_peered":"2024-03-23T23:42:36.724622+0000","last_clean":"2024-03-23T23:42:36.724622+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:42:36.724622+0000","last_undegraded":"2024-03-23T23:42:36.724622+0000","last_fullsized":"2024-03-23T23:42:36.724622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.745438+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:42:36.745438+0000","last_peered":"2024-03-23T23:42:36.745438+0000","last_clean":"2024-03-23T23:42:36.745438+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:42:36.745438+0000","last_undegraded":"2024-03-23T23:42:36.745438+0000","last_fullsized":"2024-03-23T23:42:36.745438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.756664+0000","last_change":"2024-03-23T23:42:07.885385+0000","last_active":"2024-03-23T23:42:36.756664+0000","last_peered":"2024-03-23T23:42:36.756664+0000","last_clean":"2024-03-23T23:42:36.756664+0000","last_became_active":"2024-03-23T23:42:07.884976+0000","last_became_peered":"2024-03-23T23:42:07.884976+0000","last_unstale":"2024-03-23T23:42:36.756664+0000","last_undegraded":"2024-03-23T23:42:36.756664+0000","last_fullsized":"2024-03-23T23:42:36.756664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:40:36.593277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.754642+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:42:36.754642+0000","last_peered":"2024-03-23T23:42:36.754642+0000","last_clean":"2024-03-23T23:42:36.754642+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:42:36.754642+0000","last_undegraded":"2024-03-23T23:42:36.754642+0000","last_fullsized":"2024-03-23T23:42:36.754642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722930+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:42:36.722930+0000","last_peered":"2024-03-23T23:42:36.722930+0000","last_clean":"2024-03-23T23:42:36.722930+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:42:36.722930+0000","last_undegraded":"2024-03-23T23:42:36.722930+0000","last_fullsized":"2024-03-23T23:42:36.722930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725931+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:42:36.725931+0000","last_peered":"2024-03-23T23:42:36.725931+0000","last_clean":"2024-03-23T23:42:36.725931+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:42:36.725931+0000","last_undegraded":"2024-03-23T23:42:36.725931+0000","last_fullsized":"2024-03-23T23:42:36.725931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722612+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:42:36.722612+0000","last_peered":"2024-03-23T23:42:36.722612+0000","last_clean":"2024-03-23T23:42:36.722612+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:42:36.722612+0000","last_undegraded":"2024-03-23T23:42:36.722612+0000","last_fullsized":"2024-03-23T23:42:36.722612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742255+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:42:36.742255+0000","last_peered":"2024-03-23T23:42:36.742255+0000","last_clean":"2024-03-23T23:42:36.742255+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:42:36.742255+0000","last_undegraded":"2024-03-23T23:42:36.742255+0000","last_fullsized":"2024-03-23T23:42:36.742255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730999+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:42:36.730999+0000","last_peered":"2024-03-23T23:42:36.730999+0000","last_clean":"2024-03-23T23:42:36.730999+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:42:36.730999+0000","last_undegraded":"2024-03-23T23:42:36.730999+0000","last_fullsized":"2024-03-23T23:42:36.730999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.739325+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:42:36.739325+0000","last_peered":"2024-03-23T23:42:36.739325+0000","last_clean":"2024-03-23T23:42:36.739325+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:42:36.739325+0000","last_undegraded":"2024-03-23T23:42:36.739325+0000","last_fullsized":"2024-03-23T23:42:36.739325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748678+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:42:36.748678+0000","last_peered":"2024-03-23T23:42:36.748678+0000","last_clean":"2024-03-23T23:42:36.748678+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:42:36.748678+0000","last_undegraded":"2024-03-23T23:42:36.748678+0000","last_fullsized":"2024-03-23T23:42:36.748678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.719762+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:42:36.719762+0000","last_peered":"2024-03-23T23:42:36.719762+0000","last_clean":"2024-03-23T23:42:36.719762+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:42:36.719762+0000","last_undegraded":"2024-03-23T23:42:36.719762+0000","last_fullsized":"2024-03-23T23:42:36.719762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76271,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1511,"num_read_kb":840,"num_write":255,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228813,"data_compressed":5031,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":456997,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645363,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1117343,"data_compressed":43227,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456927,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457305,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1050802,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456644,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":527190,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":457983,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":525634,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456469,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":457343,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1049642,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71385,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69829,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71053,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:43:47.105 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T23:43:47.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.380+0000 7f02ebe00640 1 -- 172.21.15.99:0/4096010964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 msgr2=0x7f02e4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.380+0000 7f02ebe00640 1 --2- 172.21.15.99:0/4096010964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e4108a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009ca0 tx=0x7f02dc02c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.380+0000 7f02ebe00640 1 -- 172.21.15.99:0/4096010964 shutdown_connections 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 --2- 172.21.15.99:0/4096010964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e4108a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 --2- 172.21.15.99:0/4096010964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 0x7f02e4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 --2- 172.21.15.99:0/4096010964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 -- 172.21.15.99:0/4096010964 >> 172.21.15.99:0/4096010964 conn(0x7f02e40fbb50 msgr2=0x7f02e40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 -- 172.21.15.99:0/4096010964 shutdown_connections 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 -- 172.21.15.99:0/4096010964 wait complete. 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.381+0000 7f02ebe00640 1 Processor -- start 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 -- start start 2024-03-23T23:43:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 0x7f02e419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e41a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e406a720 con 0x7f02e41017d0 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e406a5a0 con 0x7f02e4102190 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02ebe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e406a8a0 con 0x7f02e41005e0 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02e9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02e9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e41a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02e9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57500/0 (socket says 172.21.15.99:57500) 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02e9400640 1 -- 172.21.15.99:0/1789989496 learned_addr learned my addr 172.21.15.99:0/1789989496 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.382+0000 7f02e8a00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 0x7f02e419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 -- 172.21.15.99:0/1789989496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 msgr2=0x7f02e41a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e41a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 -- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 msgr2=0x7f02e419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 0x7f02e419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 -- 172.21.15.99:0/1789989496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e41a17d0 con 0x7f02e41005e0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9e00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e41a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e9400640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02d800e9c0 tx=0x7f02d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800cd80 con 0x7f02e41005e0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e41a1ac0 con 0x7f02e41005e0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02d80035f0 con 0x7f02e41005e0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8010690 con 0x7f02e41005e0 2024-03-23T23:43:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.383+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e41a1fc0 con 0x7f02e41005e0 2024-03-23T23:43:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.384+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e41a2250 con 0x7f02e41005e0 2024-03-23T23:43:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.388+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f02d80037a0 con 0x7f02e41005e0 2024-03-23T23:43:47.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.389+0000 7f02e2000640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 0x7f02cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.389+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f02d80cee40 con 0x7f02e41005e0 2024-03-23T23:43:47.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.389+0000 7f02e8a00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 0x7f02cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.390+0000 7f02e8a00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 0x7f02cc0a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e419b0c0 tx=0x7f02d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:47.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.393+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02d809be60 con 0x7f02e41005e0 2024-03-23T23:43:47.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:46 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/4035352437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:46 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1966536961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:47.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:46 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:47.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:46 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/4035352437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:46 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1966536961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:47.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:46 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:47.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.640+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02e410cf30 con 0x7f02cc0a2800 2024-03-23T23:43:47.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.655+0000 7f02e2000640 1 -- 172.21.15.99:0/1789989496 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259193 (secure 0 0 0) 0x7f02e410cf30 con 0x7f02cc0a2800 2024-03-23T23:43:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:43:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 msgr2=0x7f02cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 0x7f02cc0a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02e419b0c0 tx=0x7f02d000b410 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 msgr2=0x7f02e4199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02d800e9c0 tx=0x7f02d800ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 shutdown_connections 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f02cc0a2800 0x7f02cc0a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02e4102190 0x7f02e41a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02e41017d0 0x7f02e419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 --2- 172.21.15.99:0/1789989496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e41005e0 0x7f02e4199bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 >> 172.21.15.99:0/1789989496 conn(0x7f02e40fbb50 msgr2=0x7f02e410b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 shutdown_connections 2024-03-23T23:43:47.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.660+0000 7f02ebe00640 1 -- 172.21.15.99:0/1789989496 wait complete. 2024-03-23T23:43:47.681 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-03-23T23:43:46.780875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666873,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1637,"num_read_kb":949,"num_write":438,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381676,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560148,"statfs":{"total":1151940427776,"available":1151549591552,"internally_reserved":0,"allocated":12005376,"data_stored":7470279,"data_compressed":181782,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":73,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003960"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751543+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:42:36.751543+0000","last_peered":"2024-03-23T23:42:36.751543+0000","last_clean":"2024-03-23T23:42:36.751543+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:42:36.751543+0000","last_undegraded":"2024-03-23T23:42:36.751543+0000","last_fullsized":"2024-03-23T23:42:36.751543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758825+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:42:36.758825+0000","last_peered":"2024-03-23T23:42:36.758825+0000","last_clean":"2024-03-23T23:42:36.758825+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:42:36.758825+0000","last_undegraded":"2024-03-23T23:42:36.758825+0000","last_fullsized":"2024-03-23T23:42:36.758825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761403+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:42:36.761403+0000","last_peered":"2024-03-23T23:42:36.761403+0000","last_clean":"2024-03-23T23:42:36.761403+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:42:36.761403+0000","last_undegraded":"2024-03-23T23:42:36.761403+0000","last_fullsized":"2024-03-23T23:42:36.761403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.723120+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:42:36.723120+0000","last_peered":"2024-03-23T23:42:36.723120+0000","last_clean":"2024-03-23T23:42:36.723120+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:42:36.723120+0000","last_undegraded":"2024-03-23T23:42:36.723120+0000","last_fullsized":"2024-03-23T23:42:36.723120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725071+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:42:36.725071+0000","last_peered":"2024-03-23T23:42:36.725071+0000","last_clean":"2024-03-23T23:42:36.725071+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:42:36.725071+0000","last_undegraded":"2024-03-23T23:42:36.725071+0000","last_fullsized":"2024-03-23T23:42:36.725071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724996+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:42:36.724996+0000","last_peered":"2024-03-23T23:42:36.724996+0000","last_clean":"2024-03-23T23:42:36.724996+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:42:36.724996+0000","last_undegraded":"2024-03-23T23:42:36.724996+0000","last_fullsized":"2024-03-23T23:42:36.724996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748285+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:42:36.748285+0000","last_peered":"2024-03-23T23:42:36.748285+0000","last_clean":"2024-03-23T23:42:36.748285+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:42:36.748285+0000","last_undegraded":"2024-03-23T23:42:36.748285+0000","last_fullsized":"2024-03-23T23:42:36.748285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738694+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:42:36.738694+0000","last_peered":"2024-03-23T23:42:36.738694+0000","last_clean":"2024-03-23T23:42:36.738694+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:42:36.738694+0000","last_undegraded":"2024-03-23T23:42:36.738694+0000","last_fullsized":"2024-03-23T23:42:36.738694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728477+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:42:36.728477+0000","last_peered":"2024-03-23T23:42:36.728477+0000","last_clean":"2024-03-23T23:42:36.728477+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:42:36.728477+0000","last_undegraded":"2024-03-23T23:42:36.728477+0000","last_fullsized":"2024-03-23T23:42:36.728477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724950+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:42:36.724950+0000","last_peered":"2024-03-23T23:42:36.724950+0000","last_clean":"2024-03-23T23:42:36.724950+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:42:36.724950+0000","last_undegraded":"2024-03-23T23:42:36.724950+0000","last_fullsized":"2024-03-23T23:42:36.724950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.720950+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:42:36.720950+0000","last_peered":"2024-03-23T23:42:36.720950+0000","last_clean":"2024-03-23T23:42:36.720950+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:42:36.720950+0000","last_undegraded":"2024-03-23T23:42:36.720950+0000","last_fullsized":"2024-03-23T23:42:36.720950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.715575+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:42:36.715575+0000","last_peered":"2024-03-23T23:42:36.715575+0000","last_clean":"2024-03-23T23:42:36.715575+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:42:36.715575+0000","last_undegraded":"2024-03-23T23:42:36.715575+0000","last_fullsized":"2024-03-23T23:42:36.715575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.746803+0000","last_change":"2024-03-23T23:42:07.892099+0000","last_active":"2024-03-23T23:42:36.746803+0000","last_peered":"2024-03-23T23:42:36.746803+0000","last_clean":"2024-03-23T23:42:36.746803+0000","last_became_active":"2024-03-23T23:42:07.891702+0000","last_became_peered":"2024-03-23T23:42:07.891702+0000","last_unstale":"2024-03-23T23:42:36.746803+0000","last_undegraded":"2024-03-23T23:42:36.746803+0000","last_fullsized":"2024-03-23T23:42:36.746803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:52:44.779482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760590+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:42:36.760590+0000","last_peered":"2024-03-23T23:42:36.760590+0000","last_clean":"2024-03-23T23:42:36.760590+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:42:36.760590+0000","last_undegraded":"2024-03-23T23:42:36.760590+0000","last_fullsized":"2024-03-23T23:42:36.760590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758705+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:42:36.758705+0000","last_peered":"2024-03-23T23:42:36.758705+0000","last_clean":"2024-03-23T23:42:36.758705+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:42:36.758705+0000","last_undegraded":"2024-03-23T23:42:36.758705+0000","last_fullsized":"2024-03-23T23:42:36.758705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758661+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:42:36.758661+0000","last_peered":"2024-03-23T23:42:36.758661+0000","last_clean":"2024-03-23T23:42:36.758661+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:42:36.758661+0000","last_undegraded":"2024-03-23T23:42:36.758661+0000","last_fullsized":"2024-03-23T23:42:36.758661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751658+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:42:36.751658+0000","last_peered":"2024-03-23T23:42:36.751658+0000","last_clean":"2024-03-23T23:42:36.751658+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:42:36.751658+0000","last_undegraded":"2024-03-23T23:42:36.751658+0000","last_fullsized":"2024-03-23T23:42:36.751658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749416+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:42:36.749416+0000","last_peered":"2024-03-23T23:42:36.749416+0000","last_clean":"2024-03-23T23:42:36.749416+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:42:36.749416+0000","last_undegraded":"2024-03-23T23:42:36.749416+0000","last_fullsized":"2024-03-23T23:42:36.749416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.721815+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:42:36.721815+0000","last_peered":"2024-03-23T23:42:36.721815+0000","last_clean":"2024-03-23T23:42:36.721815+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:42:36.721815+0000","last_undegraded":"2024-03-23T23:42:36.721815+0000","last_fullsized":"2024-03-23T23:42:36.721815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":180,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:41.480881+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:43:41.480881+0000","last_peered":"2024-03-23T23:43:41.480881+0000","last_clean":"2024-03-23T23:43:41.480881+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:43:41.480881+0000","last_undegraded":"2024-03-23T23:43:41.480881+0000","last_fullsized":"2024-03-23T23:43:41.480881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740593+0000","last_change":"2024-03-23T23:42:07.890419+0000","last_active":"2024-03-23T23:42:36.740593+0000","last_peered":"2024-03-23T23:42:36.740593+0000","last_clean":"2024-03-23T23:42:36.740593+0000","last_became_active":"2024-03-23T23:42:07.889998+0000","last_became_peered":"2024-03-23T23:42:07.889998+0000","last_unstale":"2024-03-23T23:42:36.740593+0000","last_undegraded":"2024-03-23T23:42:36.740593+0000","last_fullsized":"2024-03-23T23:42:36.740593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:35:13.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.672810+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:42:08.672810+0000","last_peered":"2024-03-23T23:42:08.672810+0000","last_clean":"2024-03-23T23:42:08.672810+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:42:08.672810+0000","last_undegraded":"2024-03-23T23:42:08.672810+0000","last_fullsized":"2024-03-23T23:42:08.672810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761538+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:42:36.761538+0000","last_peered":"2024-03-23T23:42:36.761538+0000","last_clean":"2024-03-23T23:42:36.761538+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:42:36.761538+0000","last_undegraded":"2024-03-23T23:42:36.761538+0000","last_fullsized":"2024-03-23T23:42:36.761538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.892818+0000","last_change":"2024-03-23T23:42:07.892818+0000","last_active":"2024-03-23T23:42:07.892818+0000","last_peered":"2024-03-23T23:42:07.892818+0000","last_clean":"2024-03-23T23:42:07.892818+0000","last_became_active":"2024-03-23T23:42:07.880207+0000","last_became_peered":"2024-03-23T23:42:07.880207+0000","last_unstale":"2024-03-23T23:42:07.892818+0000","last_undegraded":"2024-03-23T23:42:07.892818+0000","last_fullsized":"2024-03-23T23:42:07.892818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:04:54.088457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743660+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:42:36.743660+0000","last_peered":"2024-03-23T23:42:36.743660+0000","last_clean":"2024-03-23T23:42:36.743660+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:42:36.743660+0000","last_undegraded":"2024-03-23T23:42:36.743660+0000","last_fullsized":"2024-03-23T23:42:36.743660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876310+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:42:07.876310+0000","last_peered":"2024-03-23T23:42:07.876310+0000","last_clean":"2024-03-23T23:42:07.876310+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:42:07.876310+0000","last_undegraded":"2024-03-23T23:42:07.876310+0000","last_fullsized":"2024-03-23T23:42:07.876310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727449+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:42:36.727449+0000","last_peered":"2024-03-23T23:42:36.727449+0000","last_clean":"2024-03-23T23:42:36.727449+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:42:36.727449+0000","last_undegraded":"2024-03-23T23:42:36.727449+0000","last_fullsized":"2024-03-23T23:42:36.727449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":187,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.475530+0000","last_change":"2024-03-23T23:42:08.388477+0000","last_active":"2024-03-23T23:43:42.475530+0000","last_peered":"2024-03-23T23:43:42.475530+0000","last_clean":"2024-03-23T23:43:42.475530+0000","last_became_active":"2024-03-23T23:42:08.384694+0000","last_became_peered":"2024-03-23T23:42:08.384694+0000","last_unstale":"2024-03-23T23:43:42.475530+0000","last_undegraded":"2024-03-23T23:43:42.475530+0000","last_fullsized":"2024-03-23T23:43:42.475530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:13.778147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.718769+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.718769+0000","last_peered":"2024-03-23T23:42:36.718769+0000","last_clean":"2024-03-23T23:42:36.718769+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:42:36.718769+0000","last_undegraded":"2024-03-23T23:42:36.718769+0000","last_fullsized":"2024-03-23T23:42:36.718769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.882785+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:42:07.882785+0000","last_peered":"2024-03-23T23:42:07.882785+0000","last_clean":"2024-03-23T23:42:07.882785+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:42:07.882785+0000","last_undegraded":"2024-03-23T23:42:07.882785+0000","last_fullsized":"2024-03-23T23:42:07.882785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743549+0000","last_change":"2024-03-23T23:42:08.673791+0000","last_active":"2024-03-23T23:42:36.743549+0000","last_peered":"2024-03-23T23:42:36.743549+0000","last_clean":"2024-03-23T23:42:36.743549+0000","last_became_active":"2024-03-23T23:42:08.673494+0000","last_became_peered":"2024-03-23T23:42:08.673494+0000","last_unstale":"2024-03-23T23:42:36.743549+0000","last_undegraded":"2024-03-23T23:42:36.743549+0000","last_fullsized":"2024-03-23T23:42:36.743549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:22:10.209266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.877587+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:42:07.877587+0000","last_peered":"2024-03-23T23:42:07.877587+0000","last_clean":"2024-03-23T23:42:07.877587+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:42:07.877587+0000","last_undegraded":"2024-03-23T23:42:07.877587+0000","last_fullsized":"2024-03-23T23:42:07.877587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738549+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:42:36.738549+0000","last_peered":"2024-03-23T23:42:36.738549+0000","last_clean":"2024-03-23T23:42:36.738549+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:42:36.738549+0000","last_undegraded":"2024-03-23T23:42:36.738549+0000","last_fullsized":"2024-03-23T23:42:36.738549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.880640+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:42:07.880640+0000","last_peered":"2024-03-23T23:42:07.880640+0000","last_clean":"2024-03-23T23:42:07.880640+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:42:07.880640+0000","last_undegraded":"2024-03-23T23:42:07.880640+0000","last_fullsized":"2024-03-23T23:42:07.880640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.750029+0000","last_change":"2024-03-23T23:42:07.891565+0000","last_active":"2024-03-23T23:42:36.750029+0000","last_peered":"2024-03-23T23:42:36.750029+0000","last_clean":"2024-03-23T23:42:36.750029+0000","last_became_active":"2024-03-23T23:42:07.890328+0000","last_became_peered":"2024-03-23T23:42:07.890328+0000","last_unstale":"2024-03-23T23:42:36.750029+0000","last_undegraded":"2024-03-23T23:42:36.750029+0000","last_fullsized":"2024-03-23T23:42:36.750029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:44:07.467469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":322,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.444327+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:43:45.444327+0000","last_peered":"2024-03-23T23:43:45.444327+0000","last_clean":"2024-03-23T23:43:45.444327+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:43:45.444327+0000","last_undegraded":"2024-03-23T23:43:45.444327+0000","last_fullsized":"2024-03-23T23:43:45.444327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742475+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:42:36.742475+0000","last_peered":"2024-03-23T23:42:36.742475+0000","last_clean":"2024-03-23T23:42:36.742475+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:42:36.742475+0000","last_undegraded":"2024-03-23T23:42:36.742475+0000","last_fullsized":"2024-03-23T23:42:36.742475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878129+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:42:07.878129+0000","last_peered":"2024-03-23T23:42:07.878129+0000","last_clean":"2024-03-23T23:42:07.878129+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:42:07.878129+0000","last_undegraded":"2024-03-23T23:42:07.878129+0000","last_fullsized":"2024-03-23T23:42:07.878129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760824+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:42:36.760824+0000","last_peered":"2024-03-23T23:42:36.760824+0000","last_clean":"2024-03-23T23:42:36.760824+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:42:36.760824+0000","last_undegraded":"2024-03-23T23:42:36.760824+0000","last_fullsized":"2024-03-23T23:42:36.760824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881851+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:42:07.881851+0000","last_peered":"2024-03-23T23:42:07.881851+0000","last_clean":"2024-03-23T23:42:07.881851+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:42:07.881851+0000","last_undegraded":"2024-03-23T23:42:07.881851+0000","last_fullsized":"2024-03-23T23:42:07.881851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728538+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:42:36.728538+0000","last_peered":"2024-03-23T23:42:36.728538+0000","last_clean":"2024-03-23T23:42:36.728538+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:42:36.728538+0000","last_undegraded":"2024-03-23T23:42:36.728538+0000","last_fullsized":"2024-03-23T23:42:36.728538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":366,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.472694+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:43:45.472694+0000","last_peered":"2024-03-23T23:43:45.472694+0000","last_clean":"2024-03-23T23:43:45.472694+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:43:45.472694+0000","last_undegraded":"2024-03-23T23:43:45.472694+0000","last_fullsized":"2024-03-23T23:43:45.472694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":307,"num_read_kb":221,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726491+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:42:36.726491+0000","last_peered":"2024-03-23T23:42:36.726491+0000","last_clean":"2024-03-23T23:42:36.726491+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:42:36.726491+0000","last_undegraded":"2024-03-23T23:42:36.726491+0000","last_fullsized":"2024-03-23T23:42:36.726491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878343+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:42:07.878343+0000","last_peered":"2024-03-23T23:42:07.878343+0000","last_clean":"2024-03-23T23:42:07.878343+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:42:07.878343+0000","last_undegraded":"2024-03-23T23:42:07.878343+0000","last_fullsized":"2024-03-23T23:42:07.878343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740369+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:42:36.740369+0000","last_peered":"2024-03-23T23:42:36.740369+0000","last_clean":"2024-03-23T23:42:36.740369+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:42:36.740369+0000","last_undegraded":"2024-03-23T23:42:36.740369+0000","last_fullsized":"2024-03-23T23:42:36.740369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881121+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:42:07.881121+0000","last_peered":"2024-03-23T23:42:07.881121+0000","last_clean":"2024-03-23T23:42:07.881121+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:42:07.881121+0000","last_undegraded":"2024-03-23T23:42:07.881121+0000","last_fullsized":"2024-03-23T23:42:07.881121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.737057+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:42:36.737057+0000","last_peered":"2024-03-23T23:42:36.737057+0000","last_clean":"2024-03-23T23:42:36.737057+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:42:36.737057+0000","last_undegraded":"2024-03-23T23:42:36.737057+0000","last_fullsized":"2024-03-23T23:42:36.737057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.497351+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:43:42.497351+0000","last_peered":"2024-03-23T23:43:42.497351+0000","last_clean":"2024-03-23T23:43:42.497351+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:43:42.497351+0000","last_undegraded":"2024-03-23T23:43:42.497351+0000","last_fullsized":"2024-03-23T23:43:42.497351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1426,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725438+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:42:36.725438+0000","last_peered":"2024-03-23T23:42:36.725438+0000","last_clean":"2024-03-23T23:42:36.725438+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:42:36.725438+0000","last_undegraded":"2024-03-23T23:42:36.725438+0000","last_fullsized":"2024-03-23T23:42:36.725438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876027+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:42:07.876027+0000","last_peered":"2024-03-23T23:42:07.876027+0000","last_clean":"2024-03-23T23:42:07.876027+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:42:07.876027+0000","last_undegraded":"2024-03-23T23:42:07.876027+0000","last_fullsized":"2024-03-23T23:42:07.876027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726840+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:42:36.726840+0000","last_peered":"2024-03-23T23:42:36.726840+0000","last_clean":"2024-03-23T23:42:36.726840+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:42:36.726840+0000","last_undegraded":"2024-03-23T23:42:36.726840+0000","last_fullsized":"2024-03-23T23:42:36.726840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876507+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:42:07.876507+0000","last_peered":"2024-03-23T23:42:07.876507+0000","last_clean":"2024-03-23T23:42:07.876507+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:42:07.876507+0000","last_undegraded":"2024-03-23T23:42:07.876507+0000","last_fullsized":"2024-03-23T23:42:07.876507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722599+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:42:36.722599+0000","last_peered":"2024-03-23T23:42:36.722599+0000","last_clean":"2024-03-23T23:42:36.722599+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:42:36.722599+0000","last_undegraded":"2024-03-23T23:42:36.722599+0000","last_fullsized":"2024-03-23T23:42:36.722599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.888239+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:42:07.888239+0000","last_peered":"2024-03-23T23:42:07.888239+0000","last_clean":"2024-03-23T23:42:07.888239+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:42:07.888239+0000","last_undegraded":"2024-03-23T23:42:07.888239+0000","last_fullsized":"2024-03-23T23:42:07.888239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728445+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:42:36.728445+0000","last_peered":"2024-03-23T23:42:36.728445+0000","last_clean":"2024-03-23T23:42:36.728445+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:42:36.728445+0000","last_undegraded":"2024-03-23T23:42:36.728445+0000","last_fullsized":"2024-03-23T23:42:36.728445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:43.484227+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:43:43.484227+0000","last_peered":"2024-03-23T23:43:43.484227+0000","last_clean":"2024-03-23T23:43:43.484227+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:43:43.484227+0000","last_undegraded":"2024-03-23T23:43:43.484227+0000","last_fullsized":"2024-03-23T23:43:43.484227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.757784+0000","last_change":"2024-03-23T23:42:07.890902+0000","last_active":"2024-03-23T23:42:36.757784+0000","last_peered":"2024-03-23T23:42:36.757784+0000","last_clean":"2024-03-23T23:42:36.757784+0000","last_became_active":"2024-03-23T23:42:07.890514+0000","last_became_peered":"2024-03-23T23:42:07.890514+0000","last_unstale":"2024-03-23T23:42:36.757784+0000","last_undegraded":"2024-03-23T23:42:36.757784+0000","last_fullsized":"2024-03-23T23:42:36.757784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:42.343732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726405+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:42:36.726405+0000","last_peered":"2024-03-23T23:42:36.726405+0000","last_clean":"2024-03-23T23:42:36.726405+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:42:36.726405+0000","last_undegraded":"2024-03-23T23:42:36.726405+0000","last_fullsized":"2024-03-23T23:42:36.726405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'7","reported_seq":173,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:40.854661+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:43:40.854661+0000","last_peered":"2024-03-23T23:43:40.854661+0000","last_clean":"2024-03-23T23:43:40.854661+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:43:40.854661+0000","last_undegraded":"2024-03-23T23:43:40.854661+0000","last_fullsized":"2024-03-23T23:43:40.854661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729867+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:42:36.729867+0000","last_peered":"2024-03-23T23:42:36.729867+0000","last_clean":"2024-03-23T23:42:36.729867+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:42:36.729867+0000","last_undegraded":"2024-03-23T23:42:36.729867+0000","last_fullsized":"2024-03-23T23:42:36.729867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360595+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:42:08.360595+0000","last_peered":"2024-03-23T23:42:08.360595+0000","last_clean":"2024-03-23T23:42:08.360595+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:42:08.360595+0000","last_undegraded":"2024-03-23T23:42:08.360595+0000","last_fullsized":"2024-03-23T23:42:08.360595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878026+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:42:07.878026+0000","last_peered":"2024-03-23T23:42:07.878026+0000","last_clean":"2024-03-23T23:42:07.878026+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:42:07.878026+0000","last_undegraded":"2024-03-23T23:42:07.878026+0000","last_fullsized":"2024-03-23T23:42:07.878026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730425+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:42:36.730425+0000","last_peered":"2024-03-23T23:42:36.730425+0000","last_clean":"2024-03-23T23:42:36.730425+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:42:36.730425+0000","last_undegraded":"2024-03-23T23:42:36.730425+0000","last_fullsized":"2024-03-23T23:42:36.730425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.895117+0000","last_change":"2024-03-23T23:42:07.895117+0000","last_active":"2024-03-23T23:42:07.895117+0000","last_peered":"2024-03-23T23:42:07.895117+0000","last_clean":"2024-03-23T23:42:07.895117+0000","last_became_active":"2024-03-23T23:42:07.881460+0000","last_became_peered":"2024-03-23T23:42:07.881460+0000","last_unstale":"2024-03-23T23:42:07.895117+0000","last_undegraded":"2024-03-23T23:42:07.895117+0000","last_fullsized":"2024-03-23T23:42:07.895117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:07.663519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876304+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:42:07.876304+0000","last_peered":"2024-03-23T23:42:07.876304+0000","last_clean":"2024-03-23T23:42:07.876304+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:42:07.876304+0000","last_undegraded":"2024-03-23T23:42:07.876304+0000","last_fullsized":"2024-03-23T23:42:07.876304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.744135+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:42:36.744135+0000","last_peered":"2024-03-23T23:42:36.744135+0000","last_clean":"2024-03-23T23:42:36.744135+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:42:36.744135+0000","last_undegraded":"2024-03-23T23:42:36.744135+0000","last_fullsized":"2024-03-23T23:42:36.744135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:18.409637+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:43:18.409637+0000","last_peered":"2024-03-23T23:43:18.409637+0000","last_clean":"2024-03-23T23:43:18.409637+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:43:18.409637+0000","last_undegraded":"2024-03-23T23:43:18.409637+0000","last_fullsized":"2024-03-23T23:43:18.409637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724617+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:42:36.724617+0000","last_peered":"2024-03-23T23:42:36.724617+0000","last_clean":"2024-03-23T23:42:36.724617+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:42:36.724617+0000","last_undegraded":"2024-03-23T23:42:36.724617+0000","last_fullsized":"2024-03-23T23:42:36.724617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360632+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:42:08.360632+0000","last_peered":"2024-03-23T23:42:08.360632+0000","last_clean":"2024-03-23T23:42:08.360632+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:42:08.360632+0000","last_undegraded":"2024-03-23T23:42:08.360632+0000","last_fullsized":"2024-03-23T23:42:08.360632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.759993+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:42:36.759993+0000","last_peered":"2024-03-23T23:42:36.759993+0000","last_clean":"2024-03-23T23:42:36.759993+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:42:36.759993+0000","last_undegraded":"2024-03-23T23:42:36.759993+0000","last_fullsized":"2024-03-23T23:42:36.759993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.383624+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:42:08.383624+0000","last_peered":"2024-03-23T23:42:08.383624+0000","last_clean":"2024-03-23T23:42:08.383624+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:42:08.383624+0000","last_undegraded":"2024-03-23T23:42:08.383624+0000","last_fullsized":"2024-03-23T23:42:08.383624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.704210+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:42:36.704210+0000","last_peered":"2024-03-23T23:42:36.704210+0000","last_clean":"2024-03-23T23:42:36.704210+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:42:36.704210+0000","last_undegraded":"2024-03-23T23:42:36.704210+0000","last_fullsized":"2024-03-23T23:42:36.704210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.671899+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:42:08.671899+0000","last_peered":"2024-03-23T23:42:08.671899+0000","last_clean":"2024-03-23T23:42:08.671899+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:42:08.671899+0000","last_undegraded":"2024-03-23T23:42:08.671899+0000","last_fullsized":"2024-03-23T23:42:08.671899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751177+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:42:36.751177+0000","last_peered":"2024-03-23T23:42:36.751177+0000","last_clean":"2024-03-23T23:42:36.751177+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:42:36.751177+0000","last_undegraded":"2024-03-23T23:42:36.751177+0000","last_fullsized":"2024-03-23T23:42:36.751177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878466+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:42:07.878466+0000","last_peered":"2024-03-23T23:42:07.878466+0000","last_clean":"2024-03-23T23:42:07.878466+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:42:07.878466+0000","last_undegraded":"2024-03-23T23:42:07.878466+0000","last_fullsized":"2024-03-23T23:42:07.878466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.755005+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:42:36.755005+0000","last_peered":"2024-03-23T23:42:36.755005+0000","last_clean":"2024-03-23T23:42:36.755005+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:42:36.755005+0000","last_undegraded":"2024-03-23T23:42:36.755005+0000","last_fullsized":"2024-03-23T23:42:36.755005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.365111+0000","last_change":"2024-03-23T23:42:08.364753+0000","last_active":"2024-03-23T23:42:08.365111+0000","last_peered":"2024-03-23T23:42:08.365111+0000","last_clean":"2024-03-23T23:42:08.365111+0000","last_became_active":"2024-03-23T23:42:08.361512+0000","last_became_peered":"2024-03-23T23:42:08.361512+0000","last_unstale":"2024-03-23T23:42:08.365111+0000","last_undegraded":"2024-03-23T23:42:08.365111+0000","last_fullsized":"2024-03-23T23:42:08.365111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:06:40.047574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.736240+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:42:36.736240+0000","last_peered":"2024-03-23T23:42:36.736240+0000","last_clean":"2024-03-23T23:42:36.736240+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:42:36.736240+0000","last_undegraded":"2024-03-23T23:42:36.736240+0000","last_fullsized":"2024-03-23T23:42:36.736240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.441432+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:43:42.441432+0000","last_peered":"2024-03-23T23:43:42.441432+0000","last_clean":"2024-03-23T23:43:42.441432+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:43:42.441432+0000","last_undegraded":"2024-03-23T23:43:42.441432+0000","last_fullsized":"2024-03-23T23:43:42.441432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727132+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:42:36.727132+0000","last_peered":"2024-03-23T23:42:36.727132+0000","last_clean":"2024-03-23T23:42:36.727132+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:42:36.727132+0000","last_undegraded":"2024-03-23T23:42:36.727132+0000","last_fullsized":"2024-03-23T23:42:36.727132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881681+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:42:07.881681+0000","last_peered":"2024-03-23T23:42:07.881681+0000","last_clean":"2024-03-23T23:42:07.881681+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:42:07.881681+0000","last_undegraded":"2024-03-23T23:42:07.881681+0000","last_fullsized":"2024-03-23T23:42:07.881681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725297+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:42:36.725297+0000","last_peered":"2024-03-23T23:42:36.725297+0000","last_clean":"2024-03-23T23:42:36.725297+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:42:36.725297+0000","last_undegraded":"2024-03-23T23:42:36.725297+0000","last_fullsized":"2024-03-23T23:42:36.725297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878327+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:42:07.878327+0000","last_peered":"2024-03-23T23:42:07.878327+0000","last_clean":"2024-03-23T23:42:07.878327+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:42:07.878327+0000","last_undegraded":"2024-03-23T23:42:07.878327+0000","last_fullsized":"2024-03-23T23:42:07.878327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727463+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:42:36.727463+0000","last_peered":"2024-03-23T23:42:36.727463+0000","last_clean":"2024-03-23T23:42:36.727463+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:42:36.727463+0000","last_undegraded":"2024-03-23T23:42:36.727463+0000","last_fullsized":"2024-03-23T23:42:36.727463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727456+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.727456+0000","last_peered":"2024-03-23T23:42:36.727456+0000","last_clean":"2024-03-23T23:42:36.727456+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:42:36.727456+0000","last_undegraded":"2024-03-23T23:42:36.727456+0000","last_fullsized":"2024-03-23T23:42:36.727456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724622+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:42:36.724622+0000","last_peered":"2024-03-23T23:42:36.724622+0000","last_clean":"2024-03-23T23:42:36.724622+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:42:36.724622+0000","last_undegraded":"2024-03-23T23:42:36.724622+0000","last_fullsized":"2024-03-23T23:42:36.724622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.745438+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:42:36.745438+0000","last_peered":"2024-03-23T23:42:36.745438+0000","last_clean":"2024-03-23T23:42:36.745438+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:42:36.745438+0000","last_undegraded":"2024-03-23T23:42:36.745438+0000","last_fullsized":"2024-03-23T23:42:36.745438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.756664+0000","last_change":"2024-03-23T23:42:07.885385+0000","last_active":"2024-03-23T23:42:36.756664+0000","last_peered":"2024-03-23T23:42:36.756664+0000","last_clean":"2024-03-23T23:42:36.756664+0000","last_became_active":"2024-03-23T23:42:07.884976+0000","last_became_peered":"2024-03-23T23:42:07.884976+0000","last_unstale":"2024-03-23T23:42:36.756664+0000","last_undegraded":"2024-03-23T23:42:36.756664+0000","last_fullsized":"2024-03-23T23:42:36.756664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:40:36.593277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.754642+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:42:36.754642+0000","last_peered":"2024-03-23T23:42:36.754642+0000","last_clean":"2024-03-23T23:42:36.754642+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:42:36.754642+0000","last_undegraded":"2024-03-23T23:42:36.754642+0000","last_fullsized":"2024-03-23T23:42:36.754642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722930+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:42:36.722930+0000","last_peered":"2024-03-23T23:42:36.722930+0000","last_clean":"2024-03-23T23:42:36.722930+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:42:36.722930+0000","last_undegraded":"2024-03-23T23:42:36.722930+0000","last_fullsized":"2024-03-23T23:42:36.722930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725931+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:42:36.725931+0000","last_peered":"2024-03-23T23:42:36.725931+0000","last_clean":"2024-03-23T23:42:36.725931+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:42:36.725931+0000","last_undegraded":"2024-03-23T23:42:36.725931+0000","last_fullsized":"2024-03-23T23:42:36.725931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722612+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:42:36.722612+0000","last_peered":"2024-03-23T23:42:36.722612+0000","last_clean":"2024-03-23T23:42:36.722612+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:42:36.722612+0000","last_undegraded":"2024-03-23T23:42:36.722612+0000","last_fullsized":"2024-03-23T23:42:36.722612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742255+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:42:36.742255+0000","last_peered":"2024-03-23T23:42:36.742255+0000","last_clean":"2024-03-23T23:42:36.742255+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:42:36.742255+0000","last_undegraded":"2024-03-23T23:42:36.742255+0000","last_fullsized":"2024-03-23T23:42:36.742255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730999+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:42:36.730999+0000","last_peered":"2024-03-23T23:42:36.730999+0000","last_clean":"2024-03-23T23:42:36.730999+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:42:36.730999+0000","last_undegraded":"2024-03-23T23:42:36.730999+0000","last_fullsized":"2024-03-23T23:42:36.730999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.739325+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:42:36.739325+0000","last_peered":"2024-03-23T23:42:36.739325+0000","last_clean":"2024-03-23T23:42:36.739325+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:42:36.739325+0000","last_undegraded":"2024-03-23T23:42:36.739325+0000","last_fullsized":"2024-03-23T23:42:36.739325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748678+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:42:36.748678+0000","last_peered":"2024-03-23T23:42:36.748678+0000","last_clean":"2024-03-23T23:42:36.748678+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:42:36.748678+0000","last_undegraded":"2024-03-23T23:42:36.748678+0000","last_fullsized":"2024-03-23T23:42:36.748678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.719762+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:42:36.719762+0000","last_peered":"2024-03-23T23:42:36.719762+0000","last_clean":"2024-03-23T23:42:36.719762+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:42:36.719762+0000","last_undegraded":"2024-03-23T23:42:36.719762+0000","last_fullsized":"2024-03-23T23:42:36.719762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76271,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1511,"num_read_kb":840,"num_write":255,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228813,"data_compressed":5031,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":456997,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645363,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1117343,"data_compressed":43227,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456927,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457305,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1050802,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456644,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":527190,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":457983,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":525634,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456469,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":457343,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1049642,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71385,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69829,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71053,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:43:47.685 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-23T23:43:47.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T23:43:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 -- 172.21.15.99:0/2063504186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 msgr2=0x7f285c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 --2- 172.21.15.99:0/2063504186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c101ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c02c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 -- 172.21.15.99:0/2063504186 shutdown_connections 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 --2- 172.21.15.99:0/2063504186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 0x7f285c109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 --2- 172.21.15.99:0/2063504186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 0x7f285c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 --2- 172.21.15.99:0/2063504186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c101ae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 -- 172.21.15.99:0/2063504186 >> 172.21.15.99:0/2063504186 conn(0x7f285c0fcc50 msgr2=0x7f285c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 -- 172.21.15.99:0/2063504186 shutdown_connections 2024-03-23T23:43:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.928+0000 7f2862a00640 1 -- 172.21.15.99:0/2063504186 wait complete. 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.929+0000 7f2862a00640 1 Processor -- start 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.929+0000 7f2862a00640 1 -- start start 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 0x7f285c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 0x7f285c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c10d340 con 0x7f285c1028b0 2024-03-23T23:43:47.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c10d1c0 con 0x7f285c103270 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2862a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c10d4c0 con 0x7f285c1016c0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57518/0 (socket says 172.21.15.99:57518) 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 -- 172.21.15.99:0/1510169319 learned_addr learned my addr 172.21.15.99:0/1510169319 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2853400640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 0x7f285c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f2860a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 0x7f285c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 -- 172.21.15.99:0/1510169319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 msgr2=0x7f285c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 0x7f285c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 -- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 msgr2=0x7f285c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 0x7f285c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.930+0000 7f285be00640 1 -- 172.21.15.99:0/1510169319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c1a17e0 con 0x7f285c1016c0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f285be00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f284c000f80 tx=0x7f284c032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c041020 con 0x7f285c1016c0 2024-03-23T23:43:47.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c1a1a70 con 0x7f285c1016c0 2024-03-23T23:43:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f284c034cd0 con 0x7f285c1016c0 2024-03-23T23:43:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c039040 con 0x7f285c1016c0 2024-03-23T23:43:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.931+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c1a1f30 con 0x7f285c1016c0 2024-03-23T23:43:47.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.932+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c1a2100 con 0x7f285c1016c0 2024-03-23T23:43:47.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.934+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f284c044020 con 0x7f285c1016c0 2024-03-23T23:43:47.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.936+0000 7f2859600640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 0x7f28400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:47.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.937+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f284c0f08f0 con 0x7f285c1016c0 2024-03-23T23:43:47.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.937+0000 7f2853400640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 0x7f28400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:47.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.937+0000 7f2853400640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 0x7f28400a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f285c19b0d0 tx=0x7f284400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:47.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:47.939+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f284c04b020 con 0x7f285c1016c0 2024-03-23T23:43:47.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:47 smithi099 ceph-mon[31715]: pgmap v397: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:47.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:43:47.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:43:47.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:47 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.225+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 --> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f285c074ff0 con 0x7f28400a2800 2024-03-23T23:43:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.234+0000 7f2859600640 1 -- 172.21.15.99:0/1510169319 <== mgr.14152 v2:172.21.15.99:6800/3957482815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259193 (secure 0 0 0) 0x7f285c074ff0 con 0x7f28400a2800 2024-03-23T23:43:48.238 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:48.239 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-23T23:43:48.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 msgr2=0x7f28400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 0x7f28400a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f285c19b0d0 tx=0x7f284400b410 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 msgr2=0x7f285c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f284c000f80 tx=0x7f284c032ec0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 shutdown_connections 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f28400a2800 0x7f28400a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f285c103270 0x7f285c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285c1028b0 0x7f285c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 --2- 172.21.15.99:0/1510169319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f285c1016c0 0x7f285c199bc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 >> 172.21.15.99:0/1510169319 conn(0x7f285c0fcc50 msgr2=0x7f285c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.239+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 shutdown_connections 2024-03-23T23:43:48.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.240+0000 7f2862a00640 1 -- 172.21.15.99:0/1510169319 wait complete. 2024-03-23T23:43:48.257 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-03-23T23:43:46.780875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666873,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1637,"num_read_kb":949,"num_write":438,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181749,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381676,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560148,"statfs":{"total":1151940427776,"available":1151549591552,"internally_reserved":0,"allocated":12005376,"data_stored":7470279,"data_compressed":181782,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5985,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":73,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003960"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751543+0000","last_change":"2024-03-23T23:41:10.276549+0000","last_active":"2024-03-23T23:42:36.751543+0000","last_peered":"2024-03-23T23:42:36.751543+0000","last_clean":"2024-03-23T23:42:36.751543+0000","last_became_active":"2024-03-23T23:41:10.276211+0000","last_became_peered":"2024-03-23T23:41:10.276211+0000","last_unstale":"2024-03-23T23:42:36.751543+0000","last_undegraded":"2024-03-23T23:42:36.751543+0000","last_fullsized":"2024-03-23T23:42:36.751543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:03.393295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758825+0000","last_change":"2024-03-23T23:41:10.279760+0000","last_active":"2024-03-23T23:42:36.758825+0000","last_peered":"2024-03-23T23:42:36.758825+0000","last_clean":"2024-03-23T23:42:36.758825+0000","last_became_active":"2024-03-23T23:41:10.279149+0000","last_became_peered":"2024-03-23T23:41:10.279149+0000","last_unstale":"2024-03-23T23:42:36.758825+0000","last_undegraded":"2024-03-23T23:42:36.758825+0000","last_fullsized":"2024-03-23T23:42:36.758825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:05:08.281148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761403+0000","last_change":"2024-03-23T23:41:10.279056+0000","last_active":"2024-03-23T23:42:36.761403+0000","last_peered":"2024-03-23T23:42:36.761403+0000","last_clean":"2024-03-23T23:42:36.761403+0000","last_became_active":"2024-03-23T23:41:10.274338+0000","last_became_peered":"2024-03-23T23:41:10.274338+0000","last_unstale":"2024-03-23T23:42:36.761403+0000","last_undegraded":"2024-03-23T23:42:36.761403+0000","last_fullsized":"2024-03-23T23:42:36.761403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:49:57.636586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.723120+0000","last_change":"2024-03-23T23:41:10.263661+0000","last_active":"2024-03-23T23:42:36.723120+0000","last_peered":"2024-03-23T23:42:36.723120+0000","last_clean":"2024-03-23T23:42:36.723120+0000","last_became_active":"2024-03-23T23:41:10.263333+0000","last_became_peered":"2024-03-23T23:41:10.263333+0000","last_unstale":"2024-03-23T23:42:36.723120+0000","last_undegraded":"2024-03-23T23:42:36.723120+0000","last_fullsized":"2024-03-23T23:42:36.723120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:27:55.160179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725071+0000","last_change":"2024-03-23T23:41:10.276989+0000","last_active":"2024-03-23T23:42:36.725071+0000","last_peered":"2024-03-23T23:42:36.725071+0000","last_clean":"2024-03-23T23:42:36.725071+0000","last_became_active":"2024-03-23T23:41:10.276642+0000","last_became_peered":"2024-03-23T23:41:10.276642+0000","last_unstale":"2024-03-23T23:42:36.725071+0000","last_undegraded":"2024-03-23T23:42:36.725071+0000","last_fullsized":"2024-03-23T23:42:36.725071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:14.648298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724996+0000","last_change":"2024-03-23T23:41:10.256614+0000","last_active":"2024-03-23T23:42:36.724996+0000","last_peered":"2024-03-23T23:42:36.724996+0000","last_clean":"2024-03-23T23:42:36.724996+0000","last_became_active":"2024-03-23T23:41:10.256286+0000","last_became_peered":"2024-03-23T23:41:10.256286+0000","last_unstale":"2024-03-23T23:42:36.724996+0000","last_undegraded":"2024-03-23T23:42:36.724996+0000","last_fullsized":"2024-03-23T23:42:36.724996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:22:49.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748285+0000","last_change":"2024-03-23T23:41:10.259865+0000","last_active":"2024-03-23T23:42:36.748285+0000","last_peered":"2024-03-23T23:42:36.748285+0000","last_clean":"2024-03-23T23:42:36.748285+0000","last_became_active":"2024-03-23T23:41:10.259514+0000","last_became_peered":"2024-03-23T23:41:10.259514+0000","last_unstale":"2024-03-23T23:42:36.748285+0000","last_undegraded":"2024-03-23T23:42:36.748285+0000","last_fullsized":"2024-03-23T23:42:36.748285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:22:24.698463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738694+0000","last_change":"2024-03-23T23:41:10.270641+0000","last_active":"2024-03-23T23:42:36.738694+0000","last_peered":"2024-03-23T23:42:36.738694+0000","last_clean":"2024-03-23T23:42:36.738694+0000","last_became_active":"2024-03-23T23:41:10.269808+0000","last_became_peered":"2024-03-23T23:41:10.269808+0000","last_unstale":"2024-03-23T23:42:36.738694+0000","last_undegraded":"2024-03-23T23:42:36.738694+0000","last_fullsized":"2024-03-23T23:42:36.738694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:48:30.568841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728477+0000","last_change":"2024-03-23T23:41:10.257518+0000","last_active":"2024-03-23T23:42:36.728477+0000","last_peered":"2024-03-23T23:42:36.728477+0000","last_clean":"2024-03-23T23:42:36.728477+0000","last_became_active":"2024-03-23T23:41:10.257097+0000","last_became_peered":"2024-03-23T23:41:10.257097+0000","last_unstale":"2024-03-23T23:42:36.728477+0000","last_undegraded":"2024-03-23T23:42:36.728477+0000","last_fullsized":"2024-03-23T23:42:36.728477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:25.601390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724950+0000","last_change":"2024-03-23T23:41:10.255651+0000","last_active":"2024-03-23T23:42:36.724950+0000","last_peered":"2024-03-23T23:42:36.724950+0000","last_clean":"2024-03-23T23:42:36.724950+0000","last_became_active":"2024-03-23T23:41:10.255322+0000","last_became_peered":"2024-03-23T23:41:10.255322+0000","last_unstale":"2024-03-23T23:42:36.724950+0000","last_undegraded":"2024-03-23T23:42:36.724950+0000","last_fullsized":"2024-03-23T23:42:36.724950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:21.605877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.720950+0000","last_change":"2024-03-23T23:41:10.264417+0000","last_active":"2024-03-23T23:42:36.720950+0000","last_peered":"2024-03-23T23:42:36.720950+0000","last_clean":"2024-03-23T23:42:36.720950+0000","last_became_active":"2024-03-23T23:41:10.264102+0000","last_became_peered":"2024-03-23T23:41:10.264102+0000","last_unstale":"2024-03-23T23:42:36.720950+0000","last_undegraded":"2024-03-23T23:42:36.720950+0000","last_fullsized":"2024-03-23T23:42:36.720950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:11:24.331350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.715575+0000","last_change":"2024-03-23T23:41:10.264393+0000","last_active":"2024-03-23T23:42:36.715575+0000","last_peered":"2024-03-23T23:42:36.715575+0000","last_clean":"2024-03-23T23:42:36.715575+0000","last_became_active":"2024-03-23T23:41:10.264050+0000","last_became_peered":"2024-03-23T23:41:10.264050+0000","last_unstale":"2024-03-23T23:42:36.715575+0000","last_undegraded":"2024-03-23T23:42:36.715575+0000","last_fullsized":"2024-03-23T23:42:36.715575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:57:11.897949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.746803+0000","last_change":"2024-03-23T23:42:07.892099+0000","last_active":"2024-03-23T23:42:36.746803+0000","last_peered":"2024-03-23T23:42:36.746803+0000","last_clean":"2024-03-23T23:42:36.746803+0000","last_became_active":"2024-03-23T23:42:07.891702+0000","last_became_peered":"2024-03-23T23:42:07.891702+0000","last_unstale":"2024-03-23T23:42:36.746803+0000","last_undegraded":"2024-03-23T23:42:36.746803+0000","last_fullsized":"2024-03-23T23:42:36.746803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:52:44.779482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760590+0000","last_change":"2024-03-23T23:41:10.277694+0000","last_active":"2024-03-23T23:42:36.760590+0000","last_peered":"2024-03-23T23:42:36.760590+0000","last_clean":"2024-03-23T23:42:36.760590+0000","last_became_active":"2024-03-23T23:41:10.272927+0000","last_became_peered":"2024-03-23T23:41:10.272927+0000","last_unstale":"2024-03-23T23:42:36.760590+0000","last_undegraded":"2024-03-23T23:42:36.760590+0000","last_fullsized":"2024-03-23T23:42:36.760590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:23:27.126447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758705+0000","last_change":"2024-03-23T23:41:10.275828+0000","last_active":"2024-03-23T23:42:36.758705+0000","last_peered":"2024-03-23T23:42:36.758705+0000","last_clean":"2024-03-23T23:42:36.758705+0000","last_became_active":"2024-03-23T23:41:10.274888+0000","last_became_peered":"2024-03-23T23:41:10.274888+0000","last_unstale":"2024-03-23T23:42:36.758705+0000","last_undegraded":"2024-03-23T23:42:36.758705+0000","last_fullsized":"2024-03-23T23:42:36.758705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:16:49.981900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.758661+0000","last_change":"2024-03-23T23:41:10.265568+0000","last_active":"2024-03-23T23:42:36.758661+0000","last_peered":"2024-03-23T23:42:36.758661+0000","last_clean":"2024-03-23T23:42:36.758661+0000","last_became_active":"2024-03-23T23:41:10.265230+0000","last_became_peered":"2024-03-23T23:41:10.265230+0000","last_unstale":"2024-03-23T23:42:36.758661+0000","last_undegraded":"2024-03-23T23:42:36.758661+0000","last_fullsized":"2024-03-23T23:42:36.758661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:38:40.936005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751658+0000","last_change":"2024-03-23T23:41:10.286533+0000","last_active":"2024-03-23T23:42:36.751658+0000","last_peered":"2024-03-23T23:42:36.751658+0000","last_clean":"2024-03-23T23:42:36.751658+0000","last_became_active":"2024-03-23T23:41:10.286279+0000","last_became_peered":"2024-03-23T23:41:10.286279+0000","last_unstale":"2024-03-23T23:42:36.751658+0000","last_undegraded":"2024-03-23T23:42:36.751658+0000","last_fullsized":"2024-03-23T23:42:36.751658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:14:13.782649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.749416+0000","last_change":"2024-03-23T23:41:10.262560+0000","last_active":"2024-03-23T23:42:36.749416+0000","last_peered":"2024-03-23T23:42:36.749416+0000","last_clean":"2024-03-23T23:42:36.749416+0000","last_became_active":"2024-03-23T23:41:10.262067+0000","last_became_peered":"2024-03-23T23:41:10.262067+0000","last_unstale":"2024-03-23T23:42:36.749416+0000","last_undegraded":"2024-03-23T23:42:36.749416+0000","last_fullsized":"2024-03-23T23:42:36.749416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:49:32.143254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.721815+0000","last_change":"2024-03-23T23:41:10.281483+0000","last_active":"2024-03-23T23:42:36.721815+0000","last_peered":"2024-03-23T23:42:36.721815+0000","last_clean":"2024-03-23T23:42:36.721815+0000","last_became_active":"2024-03-23T23:41:10.281203+0000","last_became_peered":"2024-03-23T23:41:10.281203+0000","last_unstale":"2024-03-23T23:42:36.721815+0000","last_undegraded":"2024-03-23T23:42:36.721815+0000","last_fullsized":"2024-03-23T23:42:36.721815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:26:39.886570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":180,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:41.480881+0000","last_change":"2024-03-23T23:41:09.252605+0000","last_active":"2024-03-23T23:43:41.480881+0000","last_peered":"2024-03-23T23:43:41.480881+0000","last_clean":"2024-03-23T23:43:41.480881+0000","last_became_active":"2024-03-23T23:41:09.252282+0000","last_became_peered":"2024-03-23T23:41:09.252282+0000","last_unstale":"2024-03-23T23:43:41.480881+0000","last_undegraded":"2024-03-23T23:43:41.480881+0000","last_fullsized":"2024-03-23T23:43:41.480881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:45:12.488987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740593+0000","last_change":"2024-03-23T23:42:07.890419+0000","last_active":"2024-03-23T23:42:36.740593+0000","last_peered":"2024-03-23T23:42:36.740593+0000","last_clean":"2024-03-23T23:42:36.740593+0000","last_became_active":"2024-03-23T23:42:07.889998+0000","last_became_peered":"2024-03-23T23:42:07.889998+0000","last_unstale":"2024-03-23T23:42:36.740593+0000","last_undegraded":"2024-03-23T23:42:36.740593+0000","last_fullsized":"2024-03-23T23:42:36.740593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:35:13.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.672810+0000","last_change":"2024-03-23T23:41:09.257675+0000","last_active":"2024-03-23T23:42:08.672810+0000","last_peered":"2024-03-23T23:42:08.672810+0000","last_clean":"2024-03-23T23:42:08.672810+0000","last_became_active":"2024-03-23T23:41:09.257293+0000","last_became_peered":"2024-03-23T23:41:09.257293+0000","last_unstale":"2024-03-23T23:42:08.672810+0000","last_undegraded":"2024-03-23T23:42:08.672810+0000","last_fullsized":"2024-03-23T23:42:08.672810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.696765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.761538+0000","last_change":"2024-03-23T23:41:10.284958+0000","last_active":"2024-03-23T23:42:36.761538+0000","last_peered":"2024-03-23T23:42:36.761538+0000","last_clean":"2024-03-23T23:42:36.761538+0000","last_became_active":"2024-03-23T23:41:10.284467+0000","last_became_peered":"2024-03-23T23:41:10.284467+0000","last_unstale":"2024-03-23T23:42:36.761538+0000","last_undegraded":"2024-03-23T23:42:36.761538+0000","last_fullsized":"2024-03-23T23:42:36.761538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:46.101229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.892818+0000","last_change":"2024-03-23T23:42:07.892818+0000","last_active":"2024-03-23T23:42:07.892818+0000","last_peered":"2024-03-23T23:42:07.892818+0000","last_clean":"2024-03-23T23:42:07.892818+0000","last_became_active":"2024-03-23T23:42:07.880207+0000","last_became_peered":"2024-03-23T23:42:07.880207+0000","last_unstale":"2024-03-23T23:42:07.892818+0000","last_undegraded":"2024-03-23T23:42:07.892818+0000","last_fullsized":"2024-03-23T23:42:07.892818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:04:54.088457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743660+0000","last_change":"2024-03-23T23:41:10.274277+0000","last_active":"2024-03-23T23:42:36.743660+0000","last_peered":"2024-03-23T23:42:36.743660+0000","last_clean":"2024-03-23T23:42:36.743660+0000","last_became_active":"2024-03-23T23:41:10.273810+0000","last_became_peered":"2024-03-23T23:41:10.273810+0000","last_unstale":"2024-03-23T23:42:36.743660+0000","last_undegraded":"2024-03-23T23:42:36.743660+0000","last_fullsized":"2024-03-23T23:42:36.743660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:39:05.103634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876310+0000","last_change":"2024-03-23T23:41:09.247098+0000","last_active":"2024-03-23T23:42:07.876310+0000","last_peered":"2024-03-23T23:42:07.876310+0000","last_clean":"2024-03-23T23:42:07.876310+0000","last_became_active":"2024-03-23T23:41:09.246485+0000","last_became_peered":"2024-03-23T23:41:09.246485+0000","last_unstale":"2024-03-23T23:42:07.876310+0000","last_undegraded":"2024-03-23T23:42:07.876310+0000","last_fullsized":"2024-03-23T23:42:07.876310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:30:44.909839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727449+0000","last_change":"2024-03-23T23:41:10.258888+0000","last_active":"2024-03-23T23:42:36.727449+0000","last_peered":"2024-03-23T23:42:36.727449+0000","last_clean":"2024-03-23T23:42:36.727449+0000","last_became_active":"2024-03-23T23:41:10.258569+0000","last_became_peered":"2024-03-23T23:41:10.258569+0000","last_unstale":"2024-03-23T23:42:36.727449+0000","last_undegraded":"2024-03-23T23:42:36.727449+0000","last_fullsized":"2024-03-23T23:42:36.727449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:23:24.925787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":187,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.475530+0000","last_change":"2024-03-23T23:42:08.388477+0000","last_active":"2024-03-23T23:43:42.475530+0000","last_peered":"2024-03-23T23:43:42.475530+0000","last_clean":"2024-03-23T23:43:42.475530+0000","last_became_active":"2024-03-23T23:42:08.384694+0000","last_became_peered":"2024-03-23T23:42:08.384694+0000","last_unstale":"2024-03-23T23:43:42.475530+0000","last_undegraded":"2024-03-23T23:43:42.475530+0000","last_fullsized":"2024-03-23T23:43:42.475530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:13.778147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.718769+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.718769+0000","last_peered":"2024-03-23T23:42:36.718769+0000","last_clean":"2024-03-23T23:42:36.718769+0000","last_became_active":"2024-03-23T23:41:10.258881+0000","last_became_peered":"2024-03-23T23:41:10.258881+0000","last_unstale":"2024-03-23T23:42:36.718769+0000","last_undegraded":"2024-03-23T23:42:36.718769+0000","last_fullsized":"2024-03-23T23:42:36.718769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:39:20.260577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.882785+0000","last_change":"2024-03-23T23:41:09.246955+0000","last_active":"2024-03-23T23:42:07.882785+0000","last_peered":"2024-03-23T23:42:07.882785+0000","last_clean":"2024-03-23T23:42:07.882785+0000","last_became_active":"2024-03-23T23:41:09.246623+0000","last_became_peered":"2024-03-23T23:41:09.246623+0000","last_unstale":"2024-03-23T23:42:07.882785+0000","last_undegraded":"2024-03-23T23:42:07.882785+0000","last_fullsized":"2024-03-23T23:42:07.882785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:28:49.181457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.743549+0000","last_change":"2024-03-23T23:42:08.673791+0000","last_active":"2024-03-23T23:42:36.743549+0000","last_peered":"2024-03-23T23:42:36.743549+0000","last_clean":"2024-03-23T23:42:36.743549+0000","last_became_active":"2024-03-23T23:42:08.673494+0000","last_became_peered":"2024-03-23T23:42:08.673494+0000","last_unstale":"2024-03-23T23:42:36.743549+0000","last_undegraded":"2024-03-23T23:42:36.743549+0000","last_fullsized":"2024-03-23T23:42:36.743549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:22:10.209266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.877587+0000","last_change":"2024-03-23T23:41:09.258967+0000","last_active":"2024-03-23T23:42:07.877587+0000","last_peered":"2024-03-23T23:42:07.877587+0000","last_clean":"2024-03-23T23:42:07.877587+0000","last_became_active":"2024-03-23T23:41:09.256316+0000","last_became_peered":"2024-03-23T23:41:09.256316+0000","last_unstale":"2024-03-23T23:42:07.877587+0000","last_undegraded":"2024-03-23T23:42:07.877587+0000","last_fullsized":"2024-03-23T23:42:07.877587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:56:19.256697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.738549+0000","last_change":"2024-03-23T23:41:10.282211+0000","last_active":"2024-03-23T23:42:36.738549+0000","last_peered":"2024-03-23T23:42:36.738549+0000","last_clean":"2024-03-23T23:42:36.738549+0000","last_became_active":"2024-03-23T23:41:10.281485+0000","last_became_peered":"2024-03-23T23:41:10.281485+0000","last_unstale":"2024-03-23T23:42:36.738549+0000","last_undegraded":"2024-03-23T23:42:36.738549+0000","last_fullsized":"2024-03-23T23:42:36.738549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:12:05.169536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.880640+0000","last_change":"2024-03-23T23:41:09.250408+0000","last_active":"2024-03-23T23:42:07.880640+0000","last_peered":"2024-03-23T23:42:07.880640+0000","last_clean":"2024-03-23T23:42:07.880640+0000","last_became_active":"2024-03-23T23:41:09.250086+0000","last_became_peered":"2024-03-23T23:41:09.250086+0000","last_unstale":"2024-03-23T23:42:07.880640+0000","last_undegraded":"2024-03-23T23:42:07.880640+0000","last_fullsized":"2024-03-23T23:42:07.880640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:28:24.620875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.750029+0000","last_change":"2024-03-23T23:42:07.891565+0000","last_active":"2024-03-23T23:42:36.750029+0000","last_peered":"2024-03-23T23:42:36.750029+0000","last_clean":"2024-03-23T23:42:36.750029+0000","last_became_active":"2024-03-23T23:42:07.890328+0000","last_became_peered":"2024-03-23T23:42:07.890328+0000","last_unstale":"2024-03-23T23:42:36.750029+0000","last_undegraded":"2024-03-23T23:42:36.750029+0000","last_fullsized":"2024-03-23T23:42:36.750029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:44:07.467469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":322,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.444327+0000","last_change":"2024-03-23T23:41:09.261503+0000","last_active":"2024-03-23T23:43:45.444327+0000","last_peered":"2024-03-23T23:43:45.444327+0000","last_clean":"2024-03-23T23:43:45.444327+0000","last_became_active":"2024-03-23T23:41:09.261214+0000","last_became_peered":"2024-03-23T23:41:09.261214+0000","last_unstale":"2024-03-23T23:43:45.444327+0000","last_undegraded":"2024-03-23T23:43:45.444327+0000","last_fullsized":"2024-03-23T23:43:45.444327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:21:24.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":297,"num_read_kb":297,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742475+0000","last_change":"2024-03-23T23:41:10.268001+0000","last_active":"2024-03-23T23:42:36.742475+0000","last_peered":"2024-03-23T23:42:36.742475+0000","last_clean":"2024-03-23T23:42:36.742475+0000","last_became_active":"2024-03-23T23:41:10.267697+0000","last_became_peered":"2024-03-23T23:41:10.267697+0000","last_unstale":"2024-03-23T23:42:36.742475+0000","last_undegraded":"2024-03-23T23:42:36.742475+0000","last_fullsized":"2024-03-23T23:42:36.742475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:33.083426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878129+0000","last_change":"2024-03-23T23:41:09.249846+0000","last_active":"2024-03-23T23:42:07.878129+0000","last_peered":"2024-03-23T23:42:07.878129+0000","last_clean":"2024-03-23T23:42:07.878129+0000","last_became_active":"2024-03-23T23:41:09.249510+0000","last_became_peered":"2024-03-23T23:41:09.249510+0000","last_unstale":"2024-03-23T23:42:07.878129+0000","last_undegraded":"2024-03-23T23:42:07.878129+0000","last_fullsized":"2024-03-23T23:42:07.878129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:24:05.714328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.760824+0000","last_change":"2024-03-23T23:41:10.284112+0000","last_active":"2024-03-23T23:42:36.760824+0000","last_peered":"2024-03-23T23:42:36.760824+0000","last_clean":"2024-03-23T23:42:36.760824+0000","last_became_active":"2024-03-23T23:41:10.283766+0000","last_became_peered":"2024-03-23T23:41:10.283766+0000","last_unstale":"2024-03-23T23:42:36.760824+0000","last_undegraded":"2024-03-23T23:42:36.760824+0000","last_fullsized":"2024-03-23T23:42:36.760824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:41:13.665315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881851+0000","last_change":"2024-03-23T23:41:09.248876+0000","last_active":"2024-03-23T23:42:07.881851+0000","last_peered":"2024-03-23T23:42:07.881851+0000","last_clean":"2024-03-23T23:42:07.881851+0000","last_became_active":"2024-03-23T23:41:09.248521+0000","last_became_peered":"2024-03-23T23:41:09.248521+0000","last_unstale":"2024-03-23T23:42:07.881851+0000","last_undegraded":"2024-03-23T23:42:07.881851+0000","last_fullsized":"2024-03-23T23:42:07.881851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:24:26.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728538+0000","last_change":"2024-03-23T23:41:10.254891+0000","last_active":"2024-03-23T23:42:36.728538+0000","last_peered":"2024-03-23T23:42:36.728538+0000","last_clean":"2024-03-23T23:42:36.728538+0000","last_became_active":"2024-03-23T23:41:10.254572+0000","last_became_peered":"2024-03-23T23:41:10.254572+0000","last_unstale":"2024-03-23T23:42:36.728538+0000","last_undegraded":"2024-03-23T23:42:36.728538+0000","last_fullsized":"2024-03-23T23:42:36.728538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:17:08.254122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'39","reported_seq":366,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:45.472694+0000","last_change":"2024-03-23T23:41:09.244983+0000","last_active":"2024-03-23T23:43:45.472694+0000","last_peered":"2024-03-23T23:43:45.472694+0000","last_clean":"2024-03-23T23:43:45.472694+0000","last_became_active":"2024-03-23T23:41:09.244606+0000","last_became_peered":"2024-03-23T23:41:09.244606+0000","last_unstale":"2024-03-23T23:43:45.472694+0000","last_undegraded":"2024-03-23T23:43:45.472694+0000","last_fullsized":"2024-03-23T23:43:45.472694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:28:28.530265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":307,"num_read_kb":221,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726491+0000","last_change":"2024-03-23T23:41:10.262679+0000","last_active":"2024-03-23T23:42:36.726491+0000","last_peered":"2024-03-23T23:42:36.726491+0000","last_clean":"2024-03-23T23:42:36.726491+0000","last_became_active":"2024-03-23T23:41:10.260902+0000","last_became_peered":"2024-03-23T23:41:10.260902+0000","last_unstale":"2024-03-23T23:42:36.726491+0000","last_undegraded":"2024-03-23T23:42:36.726491+0000","last_fullsized":"2024-03-23T23:42:36.726491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:25:08.643604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878343+0000","last_change":"2024-03-23T23:41:09.251468+0000","last_active":"2024-03-23T23:42:07.878343+0000","last_peered":"2024-03-23T23:42:07.878343+0000","last_clean":"2024-03-23T23:42:07.878343+0000","last_became_active":"2024-03-23T23:41:09.250760+0000","last_became_peered":"2024-03-23T23:41:09.250760+0000","last_unstale":"2024-03-23T23:42:07.878343+0000","last_undegraded":"2024-03-23T23:42:07.878343+0000","last_fullsized":"2024-03-23T23:42:07.878343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:50:45.834254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.740369+0000","last_change":"2024-03-23T23:41:10.286432+0000","last_active":"2024-03-23T23:42:36.740369+0000","last_peered":"2024-03-23T23:42:36.740369+0000","last_clean":"2024-03-23T23:42:36.740369+0000","last_became_active":"2024-03-23T23:41:10.286046+0000","last_became_peered":"2024-03-23T23:41:10.286046+0000","last_unstale":"2024-03-23T23:42:36.740369+0000","last_undegraded":"2024-03-23T23:42:36.740369+0000","last_fullsized":"2024-03-23T23:42:36.740369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:54:56.187061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881121+0000","last_change":"2024-03-23T23:41:09.252755+0000","last_active":"2024-03-23T23:42:07.881121+0000","last_peered":"2024-03-23T23:42:07.881121+0000","last_clean":"2024-03-23T23:42:07.881121+0000","last_became_active":"2024-03-23T23:41:09.252186+0000","last_became_peered":"2024-03-23T23:41:09.252186+0000","last_unstale":"2024-03-23T23:42:07.881121+0000","last_undegraded":"2024-03-23T23:42:07.881121+0000","last_fullsized":"2024-03-23T23:42:07.881121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:44:31.009826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.737057+0000","last_change":"2024-03-23T23:41:10.277371+0000","last_active":"2024-03-23T23:42:36.737057+0000","last_peered":"2024-03-23T23:42:36.737057+0000","last_clean":"2024-03-23T23:42:36.737057+0000","last_became_active":"2024-03-23T23:41:10.276879+0000","last_became_peered":"2024-03-23T23:41:10.276879+0000","last_unstale":"2024-03-23T23:42:36.737057+0000","last_undegraded":"2024-03-23T23:42:36.737057+0000","last_fullsized":"2024-03-23T23:42:36.737057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:42:01.317283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'8","reported_seq":179,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.497351+0000","last_change":"2024-03-23T23:41:09.245961+0000","last_active":"2024-03-23T23:43:42.497351+0000","last_peered":"2024-03-23T23:43:42.497351+0000","last_clean":"2024-03-23T23:43:42.497351+0000","last_became_active":"2024-03-23T23:41:09.245597+0000","last_became_peered":"2024-03-23T23:41:09.245597+0000","last_unstale":"2024-03-23T23:43:42.497351+0000","last_undegraded":"2024-03-23T23:43:42.497351+0000","last_fullsized":"2024-03-23T23:43:42.497351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:40:35.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1426,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725438+0000","last_change":"2024-03-23T23:41:10.256844+0000","last_active":"2024-03-23T23:42:36.725438+0000","last_peered":"2024-03-23T23:42:36.725438+0000","last_clean":"2024-03-23T23:42:36.725438+0000","last_became_active":"2024-03-23T23:41:10.256461+0000","last_became_peered":"2024-03-23T23:41:10.256461+0000","last_unstale":"2024-03-23T23:42:36.725438+0000","last_undegraded":"2024-03-23T23:42:36.725438+0000","last_fullsized":"2024-03-23T23:42:36.725438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:06.723671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876027+0000","last_change":"2024-03-23T23:41:09.252407+0000","last_active":"2024-03-23T23:42:07.876027+0000","last_peered":"2024-03-23T23:42:07.876027+0000","last_clean":"2024-03-23T23:42:07.876027+0000","last_became_active":"2024-03-23T23:41:09.252108+0000","last_became_peered":"2024-03-23T23:41:09.252108+0000","last_unstale":"2024-03-23T23:42:07.876027+0000","last_undegraded":"2024-03-23T23:42:07.876027+0000","last_fullsized":"2024-03-23T23:42:07.876027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:45.137734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726840+0000","last_change":"2024-03-23T23:41:10.259841+0000","last_active":"2024-03-23T23:42:36.726840+0000","last_peered":"2024-03-23T23:42:36.726840+0000","last_clean":"2024-03-23T23:42:36.726840+0000","last_became_active":"2024-03-23T23:41:10.259463+0000","last_became_peered":"2024-03-23T23:41:10.259463+0000","last_unstale":"2024-03-23T23:42:36.726840+0000","last_undegraded":"2024-03-23T23:42:36.726840+0000","last_fullsized":"2024-03-23T23:42:36.726840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:48:08.176339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876507+0000","last_change":"2024-03-23T23:41:09.246113+0000","last_active":"2024-03-23T23:42:07.876507+0000","last_peered":"2024-03-23T23:42:07.876507+0000","last_clean":"2024-03-23T23:42:07.876507+0000","last_became_active":"2024-03-23T23:41:09.245626+0000","last_became_peered":"2024-03-23T23:41:09.245626+0000","last_unstale":"2024-03-23T23:42:07.876507+0000","last_undegraded":"2024-03-23T23:42:07.876507+0000","last_fullsized":"2024-03-23T23:42:07.876507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:54:53.697953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722599+0000","last_change":"2024-03-23T23:41:10.258614+0000","last_active":"2024-03-23T23:42:36.722599+0000","last_peered":"2024-03-23T23:42:36.722599+0000","last_clean":"2024-03-23T23:42:36.722599+0000","last_became_active":"2024-03-23T23:41:10.258291+0000","last_became_peered":"2024-03-23T23:41:10.258291+0000","last_unstale":"2024-03-23T23:42:36.722599+0000","last_undegraded":"2024-03-23T23:42:36.722599+0000","last_fullsized":"2024-03-23T23:42:36.722599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.852897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.888239+0000","last_change":"2024-03-23T23:41:09.251964+0000","last_active":"2024-03-23T23:42:07.888239+0000","last_peered":"2024-03-23T23:42:07.888239+0000","last_clean":"2024-03-23T23:42:07.888239+0000","last_became_active":"2024-03-23T23:41:09.251487+0000","last_became_peered":"2024-03-23T23:41:09.251487+0000","last_unstale":"2024-03-23T23:42:07.888239+0000","last_undegraded":"2024-03-23T23:42:07.888239+0000","last_fullsized":"2024-03-23T23:42:07.888239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:16:40.118022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.728445+0000","last_change":"2024-03-23T23:41:10.281492+0000","last_active":"2024-03-23T23:42:36.728445+0000","last_peered":"2024-03-23T23:42:36.728445+0000","last_clean":"2024-03-23T23:42:36.728445+0000","last_became_active":"2024-03-23T23:41:10.281216+0000","last_became_peered":"2024-03-23T23:41:10.281216+0000","last_unstale":"2024-03-23T23:42:36.728445+0000","last_undegraded":"2024-03-23T23:42:36.728445+0000","last_fullsized":"2024-03-23T23:42:36.728445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:06:45.486369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:43.484227+0000","last_change":"2024-03-23T23:41:09.247445+0000","last_active":"2024-03-23T23:43:43.484227+0000","last_peered":"2024-03-23T23:43:43.484227+0000","last_clean":"2024-03-23T23:43:43.484227+0000","last_became_active":"2024-03-23T23:41:09.247127+0000","last_became_peered":"2024-03-23T23:41:09.247127+0000","last_unstale":"2024-03-23T23:43:43.484227+0000","last_undegraded":"2024-03-23T23:43:43.484227+0000","last_fullsized":"2024-03-23T23:43:43.484227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:33:53.420444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.757784+0000","last_change":"2024-03-23T23:42:07.890902+0000","last_active":"2024-03-23T23:42:36.757784+0000","last_peered":"2024-03-23T23:42:36.757784+0000","last_clean":"2024-03-23T23:42:36.757784+0000","last_became_active":"2024-03-23T23:42:07.890514+0000","last_became_peered":"2024-03-23T23:42:07.890514+0000","last_unstale":"2024-03-23T23:42:36.757784+0000","last_undegraded":"2024-03-23T23:42:36.757784+0000","last_fullsized":"2024-03-23T23:42:36.757784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:42.343732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.726405+0000","last_change":"2024-03-23T23:41:10.255875+0000","last_active":"2024-03-23T23:42:36.726405+0000","last_peered":"2024-03-23T23:42:36.726405+0000","last_clean":"2024-03-23T23:42:36.726405+0000","last_became_active":"2024-03-23T23:41:10.255529+0000","last_became_peered":"2024-03-23T23:41:10.255529+0000","last_unstale":"2024-03-23T23:42:36.726405+0000","last_undegraded":"2024-03-23T23:42:36.726405+0000","last_fullsized":"2024-03-23T23:42:36.726405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:44.690424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'7","reported_seq":173,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:40.854661+0000","last_change":"2024-03-23T23:41:09.254425+0000","last_active":"2024-03-23T23:43:40.854661+0000","last_peered":"2024-03-23T23:43:40.854661+0000","last_clean":"2024-03-23T23:43:40.854661+0000","last_became_active":"2024-03-23T23:41:09.253518+0000","last_became_peered":"2024-03-23T23:41:09.253518+0000","last_unstale":"2024-03-23T23:43:40.854661+0000","last_undegraded":"2024-03-23T23:43:40.854661+0000","last_fullsized":"2024-03-23T23:43:40.854661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:10.213480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.729867+0000","last_change":"2024-03-23T23:41:10.266051+0000","last_active":"2024-03-23T23:42:36.729867+0000","last_peered":"2024-03-23T23:42:36.729867+0000","last_clean":"2024-03-23T23:42:36.729867+0000","last_became_active":"2024-03-23T23:41:10.265074+0000","last_became_peered":"2024-03-23T23:41:10.265074+0000","last_unstale":"2024-03-23T23:42:36.729867+0000","last_undegraded":"2024-03-23T23:42:36.729867+0000","last_fullsized":"2024-03-23T23:42:36.729867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:43:53.447322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360595+0000","last_change":"2024-03-23T23:41:09.263989+0000","last_active":"2024-03-23T23:42:08.360595+0000","last_peered":"2024-03-23T23:42:08.360595+0000","last_clean":"2024-03-23T23:42:08.360595+0000","last_became_active":"2024-03-23T23:41:09.263612+0000","last_became_peered":"2024-03-23T23:41:09.263612+0000","last_unstale":"2024-03-23T23:42:08.360595+0000","last_undegraded":"2024-03-23T23:42:08.360595+0000","last_fullsized":"2024-03-23T23:42:08.360595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:00:37.563942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878026+0000","last_change":"2024-03-23T23:39:49.530204+0000","last_active":"2024-03-23T23:42:07.878026+0000","last_peered":"2024-03-23T23:42:07.878026+0000","last_clean":"2024-03-23T23:42:07.878026+0000","last_became_active":"2024-03-23T23:39:49.529757+0000","last_became_peered":"2024-03-23T23:39:49.529757+0000","last_unstale":"2024-03-23T23:42:07.878026+0000","last_undegraded":"2024-03-23T23:42:07.878026+0000","last_fullsized":"2024-03-23T23:42:07.878026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:35:55.293864+0000","last_clean_scrub_stamp":"2024-03-23T23:35:55.293864+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:27:27.581847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730425+0000","last_change":"2024-03-23T23:41:10.276964+0000","last_active":"2024-03-23T23:42:36.730425+0000","last_peered":"2024-03-23T23:42:36.730425+0000","last_clean":"2024-03-23T23:42:36.730425+0000","last_became_active":"2024-03-23T23:41:10.276595+0000","last_became_peered":"2024-03-23T23:41:10.276595+0000","last_unstale":"2024-03-23T23:42:36.730425+0000","last_undegraded":"2024-03-23T23:42:36.730425+0000","last_fullsized":"2024-03-23T23:42:36.730425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:52:13.442116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.895117+0000","last_change":"2024-03-23T23:42:07.895117+0000","last_active":"2024-03-23T23:42:07.895117+0000","last_peered":"2024-03-23T23:42:07.895117+0000","last_clean":"2024-03-23T23:42:07.895117+0000","last_became_active":"2024-03-23T23:42:07.881460+0000","last_became_peered":"2024-03-23T23:42:07.881460+0000","last_unstale":"2024-03-23T23:42:07.895117+0000","last_undegraded":"2024-03-23T23:42:07.895117+0000","last_fullsized":"2024-03-23T23:42:07.895117+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:51:07.663519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.876304+0000","last_change":"2024-03-23T23:41:09.253581+0000","last_active":"2024-03-23T23:42:07.876304+0000","last_peered":"2024-03-23T23:42:07.876304+0000","last_clean":"2024-03-23T23:42:07.876304+0000","last_became_active":"2024-03-23T23:41:09.253163+0000","last_became_peered":"2024-03-23T23:41:09.253163+0000","last_unstale":"2024-03-23T23:42:07.876304+0000","last_undegraded":"2024-03-23T23:42:07.876304+0000","last_fullsized":"2024-03-23T23:42:07.876304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:52:58.246415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.744135+0000","last_change":"2024-03-23T23:41:10.279614+0000","last_active":"2024-03-23T23:42:36.744135+0000","last_peered":"2024-03-23T23:42:36.744135+0000","last_clean":"2024-03-23T23:42:36.744135+0000","last_became_active":"2024-03-23T23:41:10.279271+0000","last_became_peered":"2024-03-23T23:41:10.279271+0000","last_unstale":"2024-03-23T23:42:36.744135+0000","last_undegraded":"2024-03-23T23:42:36.744135+0000","last_fullsized":"2024-03-23T23:42:36.744135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:17:54.041521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:18.409637+0000","last_change":"2024-03-23T23:41:09.257550+0000","last_active":"2024-03-23T23:43:18.409637+0000","last_peered":"2024-03-23T23:43:18.409637+0000","last_clean":"2024-03-23T23:43:18.409637+0000","last_became_active":"2024-03-23T23:41:09.256186+0000","last_became_peered":"2024-03-23T23:41:09.256186+0000","last_unstale":"2024-03-23T23:43:18.409637+0000","last_undegraded":"2024-03-23T23:43:18.409637+0000","last_fullsized":"2024-03-23T23:43:18.409637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:05.567841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724617+0000","last_change":"2024-03-23T23:41:10.274880+0000","last_active":"2024-03-23T23:42:36.724617+0000","last_peered":"2024-03-23T23:42:36.724617+0000","last_clean":"2024-03-23T23:42:36.724617+0000","last_became_active":"2024-03-23T23:41:10.272938+0000","last_became_peered":"2024-03-23T23:41:10.272938+0000","last_unstale":"2024-03-23T23:42:36.724617+0000","last_undegraded":"2024-03-23T23:42:36.724617+0000","last_fullsized":"2024-03-23T23:42:36.724617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:15:21.933065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.360632+0000","last_change":"2024-03-23T23:41:09.248397+0000","last_active":"2024-03-23T23:42:08.360632+0000","last_peered":"2024-03-23T23:42:08.360632+0000","last_clean":"2024-03-23T23:42:08.360632+0000","last_became_active":"2024-03-23T23:41:09.248063+0000","last_became_peered":"2024-03-23T23:41:09.248063+0000","last_unstale":"2024-03-23T23:42:08.360632+0000","last_undegraded":"2024-03-23T23:42:08.360632+0000","last_fullsized":"2024-03-23T23:42:08.360632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:30:56.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.759993+0000","last_change":"2024-03-23T23:41:10.278901+0000","last_active":"2024-03-23T23:42:36.759993+0000","last_peered":"2024-03-23T23:42:36.759993+0000","last_clean":"2024-03-23T23:42:36.759993+0000","last_became_active":"2024-03-23T23:41:10.275921+0000","last_became_peered":"2024-03-23T23:41:10.275921+0000","last_unstale":"2024-03-23T23:42:36.759993+0000","last_undegraded":"2024-03-23T23:42:36.759993+0000","last_fullsized":"2024-03-23T23:42:36.759993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:57:15.394931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.383624+0000","last_change":"2024-03-23T23:41:09.248183+0000","last_active":"2024-03-23T23:42:08.383624+0000","last_peered":"2024-03-23T23:42:08.383624+0000","last_clean":"2024-03-23T23:42:08.383624+0000","last_became_active":"2024-03-23T23:41:09.247846+0000","last_became_peered":"2024-03-23T23:41:09.247846+0000","last_unstale":"2024-03-23T23:42:08.383624+0000","last_undegraded":"2024-03-23T23:42:08.383624+0000","last_fullsized":"2024-03-23T23:42:08.383624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:26:54.520381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.704210+0000","last_change":"2024-03-23T23:41:10.255918+0000","last_active":"2024-03-23T23:42:36.704210+0000","last_peered":"2024-03-23T23:42:36.704210+0000","last_clean":"2024-03-23T23:42:36.704210+0000","last_became_active":"2024-03-23T23:41:10.255515+0000","last_became_peered":"2024-03-23T23:41:10.255515+0000","last_unstale":"2024-03-23T23:42:36.704210+0000","last_undegraded":"2024-03-23T23:42:36.704210+0000","last_fullsized":"2024-03-23T23:42:36.704210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:27:20.262442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.671899+0000","last_change":"2024-03-23T23:41:09.255162+0000","last_active":"2024-03-23T23:42:08.671899+0000","last_peered":"2024-03-23T23:42:08.671899+0000","last_clean":"2024-03-23T23:42:08.671899+0000","last_became_active":"2024-03-23T23:41:09.252154+0000","last_became_peered":"2024-03-23T23:41:09.252154+0000","last_unstale":"2024-03-23T23:42:08.671899+0000","last_undegraded":"2024-03-23T23:42:08.671899+0000","last_fullsized":"2024-03-23T23:42:08.671899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:42.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.751177+0000","last_change":"2024-03-23T23:41:10.276523+0000","last_active":"2024-03-23T23:42:36.751177+0000","last_peered":"2024-03-23T23:42:36.751177+0000","last_clean":"2024-03-23T23:42:36.751177+0000","last_became_active":"2024-03-23T23:41:10.276193+0000","last_became_peered":"2024-03-23T23:41:10.276193+0000","last_unstale":"2024-03-23T23:42:36.751177+0000","last_undegraded":"2024-03-23T23:42:36.751177+0000","last_fullsized":"2024-03-23T23:42:36.751177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:01:33.194431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878466+0000","last_change":"2024-03-23T23:41:09.252083+0000","last_active":"2024-03-23T23:42:07.878466+0000","last_peered":"2024-03-23T23:42:07.878466+0000","last_clean":"2024-03-23T23:42:07.878466+0000","last_became_active":"2024-03-23T23:41:09.250140+0000","last_became_peered":"2024-03-23T23:41:09.250140+0000","last_unstale":"2024-03-23T23:42:07.878466+0000","last_undegraded":"2024-03-23T23:42:07.878466+0000","last_fullsized":"2024-03-23T23:42:07.878466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:02:12.677813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.755005+0000","last_change":"2024-03-23T23:41:10.287126+0000","last_active":"2024-03-23T23:42:36.755005+0000","last_peered":"2024-03-23T23:42:36.755005+0000","last_clean":"2024-03-23T23:42:36.755005+0000","last_became_active":"2024-03-23T23:41:10.286665+0000","last_became_peered":"2024-03-23T23:41:10.286665+0000","last_unstale":"2024-03-23T23:42:36.755005+0000","last_undegraded":"2024-03-23T23:42:36.755005+0000","last_fullsized":"2024-03-23T23:42:36.755005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:41:08.329454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:08.365111+0000","last_change":"2024-03-23T23:42:08.364753+0000","last_active":"2024-03-23T23:42:08.365111+0000","last_peered":"2024-03-23T23:42:08.365111+0000","last_clean":"2024-03-23T23:42:08.365111+0000","last_became_active":"2024-03-23T23:42:08.361512+0000","last_became_peered":"2024-03-23T23:42:08.361512+0000","last_unstale":"2024-03-23T23:42:08.365111+0000","last_undegraded":"2024-03-23T23:42:08.365111+0000","last_fullsized":"2024-03-23T23:42:08.365111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:06:40.047574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.736240+0000","last_change":"2024-03-23T23:41:10.279465+0000","last_active":"2024-03-23T23:42:36.736240+0000","last_peered":"2024-03-23T23:42:36.736240+0000","last_clean":"2024-03-23T23:42:36.736240+0000","last_became_active":"2024-03-23T23:41:10.279109+0000","last_became_peered":"2024-03-23T23:41:10.279109+0000","last_unstale":"2024-03-23T23:42:36.736240+0000","last_undegraded":"2024-03-23T23:42:36.736240+0000","last_fullsized":"2024-03-23T23:42:36.736240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:16:50.857440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":174,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:43:42.441432+0000","last_change":"2024-03-23T23:41:09.246149+0000","last_active":"2024-03-23T23:43:42.441432+0000","last_peered":"2024-03-23T23:43:42.441432+0000","last_clean":"2024-03-23T23:43:42.441432+0000","last_became_active":"2024-03-23T23:41:09.245795+0000","last_became_peered":"2024-03-23T23:41:09.245795+0000","last_unstale":"2024-03-23T23:43:42.441432+0000","last_undegraded":"2024-03-23T23:43:42.441432+0000","last_fullsized":"2024-03-23T23:43:42.441432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:04:44.690708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727132+0000","last_change":"2024-03-23T23:41:10.258342+0000","last_active":"2024-03-23T23:42:36.727132+0000","last_peered":"2024-03-23T23:42:36.727132+0000","last_clean":"2024-03-23T23:42:36.727132+0000","last_became_active":"2024-03-23T23:41:10.258008+0000","last_became_peered":"2024-03-23T23:41:10.258008+0000","last_unstale":"2024-03-23T23:42:36.727132+0000","last_undegraded":"2024-03-23T23:42:36.727132+0000","last_fullsized":"2024-03-23T23:42:36.727132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T09:14:44.616210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.881681+0000","last_change":"2024-03-23T23:41:09.254680+0000","last_active":"2024-03-23T23:42:07.881681+0000","last_peered":"2024-03-23T23:42:07.881681+0000","last_clean":"2024-03-23T23:42:07.881681+0000","last_became_active":"2024-03-23T23:41:09.254320+0000","last_became_peered":"2024-03-23T23:41:09.254320+0000","last_unstale":"2024-03-23T23:42:07.881681+0000","last_undegraded":"2024-03-23T23:42:07.881681+0000","last_fullsized":"2024-03-23T23:42:07.881681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:12:57.771972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725297+0000","last_change":"2024-03-23T23:41:10.273209+0000","last_active":"2024-03-23T23:42:36.725297+0000","last_peered":"2024-03-23T23:42:36.725297+0000","last_clean":"2024-03-23T23:42:36.725297+0000","last_became_active":"2024-03-23T23:41:10.272774+0000","last_became_peered":"2024-03-23T23:41:10.272774+0000","last_unstale":"2024-03-23T23:42:36.725297+0000","last_undegraded":"2024-03-23T23:42:36.725297+0000","last_fullsized":"2024-03-23T23:42:36.725297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:55:23.369734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:07.878327+0000","last_change":"2024-03-23T23:41:09.262468+0000","last_active":"2024-03-23T23:42:07.878327+0000","last_peered":"2024-03-23T23:42:07.878327+0000","last_clean":"2024-03-23T23:42:07.878327+0000","last_became_active":"2024-03-23T23:41:09.262107+0000","last_became_peered":"2024-03-23T23:41:09.262107+0000","last_unstale":"2024-03-23T23:42:07.878327+0000","last_undegraded":"2024-03-23T23:42:07.878327+0000","last_fullsized":"2024-03-23T23:42:07.878327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:08.152992+0000","last_clean_scrub_stamp":"2024-03-23T23:41:08.152992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:11.572588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727463+0000","last_change":"2024-03-23T23:41:10.262049+0000","last_active":"2024-03-23T23:42:36.727463+0000","last_peered":"2024-03-23T23:42:36.727463+0000","last_clean":"2024-03-23T23:42:36.727463+0000","last_became_active":"2024-03-23T23:41:10.261652+0000","last_became_peered":"2024-03-23T23:41:10.261652+0000","last_unstale":"2024-03-23T23:42:36.727463+0000","last_undegraded":"2024-03-23T23:42:36.727463+0000","last_fullsized":"2024-03-23T23:42:36.727463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:53:00.339061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.727456+0000","last_change":"2024-03-23T23:41:10.259921+0000","last_active":"2024-03-23T23:42:36.727456+0000","last_peered":"2024-03-23T23:42:36.727456+0000","last_clean":"2024-03-23T23:42:36.727456+0000","last_became_active":"2024-03-23T23:41:10.259581+0000","last_became_peered":"2024-03-23T23:41:10.259581+0000","last_unstale":"2024-03-23T23:42:36.727456+0000","last_undegraded":"2024-03-23T23:42:36.727456+0000","last_fullsized":"2024-03-23T23:42:36.727456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:57:23.274544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.724622+0000","last_change":"2024-03-23T23:41:10.263338+0000","last_active":"2024-03-23T23:42:36.724622+0000","last_peered":"2024-03-23T23:42:36.724622+0000","last_clean":"2024-03-23T23:42:36.724622+0000","last_became_active":"2024-03-23T23:41:10.262343+0000","last_became_peered":"2024-03-23T23:41:10.262343+0000","last_unstale":"2024-03-23T23:42:36.724622+0000","last_undegraded":"2024-03-23T23:42:36.724622+0000","last_fullsized":"2024-03-23T23:42:36.724622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:16:39.801110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.745438+0000","last_change":"2024-03-23T23:41:10.276463+0000","last_active":"2024-03-23T23:42:36.745438+0000","last_peered":"2024-03-23T23:42:36.745438+0000","last_clean":"2024-03-23T23:42:36.745438+0000","last_became_active":"2024-03-23T23:41:10.276071+0000","last_became_peered":"2024-03-23T23:41:10.276071+0000","last_unstale":"2024-03-23T23:42:36.745438+0000","last_undegraded":"2024-03-23T23:42:36.745438+0000","last_fullsized":"2024-03-23T23:42:36.745438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:28:53.471366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.756664+0000","last_change":"2024-03-23T23:42:07.885385+0000","last_active":"2024-03-23T23:42:36.756664+0000","last_peered":"2024-03-23T23:42:36.756664+0000","last_clean":"2024-03-23T23:42:36.756664+0000","last_became_active":"2024-03-23T23:42:07.884976+0000","last_became_peered":"2024-03-23T23:42:07.884976+0000","last_unstale":"2024-03-23T23:42:36.756664+0000","last_undegraded":"2024-03-23T23:42:36.756664+0000","last_fullsized":"2024-03-23T23:42:36.756664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T10:40:36.593277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.754642+0000","last_change":"2024-03-23T23:41:10.284567+0000","last_active":"2024-03-23T23:42:36.754642+0000","last_peered":"2024-03-23T23:42:36.754642+0000","last_clean":"2024-03-23T23:42:36.754642+0000","last_became_active":"2024-03-23T23:41:10.284255+0000","last_became_peered":"2024-03-23T23:41:10.284255+0000","last_unstale":"2024-03-23T23:42:36.754642+0000","last_undegraded":"2024-03-23T23:42:36.754642+0000","last_fullsized":"2024-03-23T23:42:36.754642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T04:35:58.031350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722930+0000","last_change":"2024-03-23T23:41:10.263888+0000","last_active":"2024-03-23T23:42:36.722930+0000","last_peered":"2024-03-23T23:42:36.722930+0000","last_clean":"2024-03-23T23:42:36.722930+0000","last_became_active":"2024-03-23T23:41:10.263513+0000","last_became_peered":"2024-03-23T23:41:10.263513+0000","last_unstale":"2024-03-23T23:42:36.722930+0000","last_undegraded":"2024-03-23T23:42:36.722930+0000","last_fullsized":"2024-03-23T23:42:36.722930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:16:10.924516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.725931+0000","last_change":"2024-03-23T23:41:10.258690+0000","last_active":"2024-03-23T23:42:36.725931+0000","last_peered":"2024-03-23T23:42:36.725931+0000","last_clean":"2024-03-23T23:42:36.725931+0000","last_became_active":"2024-03-23T23:41:10.258329+0000","last_became_peered":"2024-03-23T23:41:10.258329+0000","last_unstale":"2024-03-23T23:42:36.725931+0000","last_undegraded":"2024-03-23T23:42:36.725931+0000","last_fullsized":"2024-03-23T23:42:36.725931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T07:45:19.282485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.722612+0000","last_change":"2024-03-23T23:41:10.282631+0000","last_active":"2024-03-23T23:42:36.722612+0000","last_peered":"2024-03-23T23:42:36.722612+0000","last_clean":"2024-03-23T23:42:36.722612+0000","last_became_active":"2024-03-23T23:41:10.280990+0000","last_became_peered":"2024-03-23T23:41:10.280990+0000","last_unstale":"2024-03-23T23:42:36.722612+0000","last_undegraded":"2024-03-23T23:42:36.722612+0000","last_fullsized":"2024-03-23T23:42:36.722612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T11:40:21.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.742255+0000","last_change":"2024-03-23T23:41:10.268659+0000","last_active":"2024-03-23T23:42:36.742255+0000","last_peered":"2024-03-23T23:42:36.742255+0000","last_clean":"2024-03-23T23:42:36.742255+0000","last_became_active":"2024-03-23T23:41:10.268360+0000","last_became_peered":"2024-03-23T23:41:10.268360+0000","last_unstale":"2024-03-23T23:42:36.742255+0000","last_undegraded":"2024-03-23T23:42:36.742255+0000","last_fullsized":"2024-03-23T23:42:36.742255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:51:44.563706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.730999+0000","last_change":"2024-03-23T23:41:10.270310+0000","last_active":"2024-03-23T23:42:36.730999+0000","last_peered":"2024-03-23T23:42:36.730999+0000","last_clean":"2024-03-23T23:42:36.730999+0000","last_became_active":"2024-03-23T23:41:10.268992+0000","last_became_peered":"2024-03-23T23:41:10.268992+0000","last_unstale":"2024-03-23T23:42:36.730999+0000","last_undegraded":"2024-03-23T23:42:36.730999+0000","last_fullsized":"2024-03-23T23:42:36.730999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T06:35:42.207876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.739325+0000","last_change":"2024-03-23T23:41:10.267035+0000","last_active":"2024-03-23T23:42:36.739325+0000","last_peered":"2024-03-23T23:42:36.739325+0000","last_clean":"2024-03-23T23:42:36.739325+0000","last_became_active":"2024-03-23T23:41:10.266666+0000","last_became_peered":"2024-03-23T23:41:10.266666+0000","last_unstale":"2024-03-23T23:42:36.739325+0000","last_undegraded":"2024-03-23T23:42:36.739325+0000","last_fullsized":"2024-03-23T23:42:36.739325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T05:13:56.096746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.748678+0000","last_change":"2024-03-23T23:41:10.287380+0000","last_active":"2024-03-23T23:42:36.748678+0000","last_peered":"2024-03-23T23:42:36.748678+0000","last_clean":"2024-03-23T23:42:36.748678+0000","last_became_active":"2024-03-23T23:41:10.286990+0000","last_became_peered":"2024-03-23T23:41:10.286990+0000","last_unstale":"2024-03-23T23:42:36.748678+0000","last_undegraded":"2024-03-23T23:42:36.748678+0000","last_fullsized":"2024-03-23T23:42:36.748678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:44:38.662438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T23:42:36.719762+0000","last_change":"2024-03-23T23:41:10.262033+0000","last_active":"2024-03-23T23:42:36.719762+0000","last_peered":"2024-03-23T23:42:36.719762+0000","last_clean":"2024-03-23T23:42:36.719762+0000","last_became_active":"2024-03-23T23:41:10.261694+0000","last_became_peered":"2024-03-23T23:41:10.261694+0000","last_unstale":"2024-03-23T23:42:36.719762+0000","last_undegraded":"2024-03-23T23:42:36.719762+0000","last_fullsized":"2024-03-23T23:42:36.719762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T23:41:09.219803+0000","last_clean_scrub_stamp":"2024-03-23T23:41:09.219803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T08:33:00.737193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281153,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76271,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1511,"num_read_kb":840,"num_write":255,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1013,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228813,"data_compressed":5031,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":456997,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645363,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1117343,"data_compressed":43227,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":456927,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457305,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1050802,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495689,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456644,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":527190,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":457983,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":525634,"data_compressed":7466,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456469,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":457343,"data_compressed":5789,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":1049642,"data_compressed":41550,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34116,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35761,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71385,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69829,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71053,"data_compressed":1677,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T23:43:48.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:47 smithi149 ceph-mon[36083]: pgmap v397: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:43:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:43:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:47 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:48.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:47 smithi174 ceph-mon[37002]: pgmap v397: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:43:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:43:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:47 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:43:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.489+0000 7f0acb200640 1 -- 172.21.15.99:0/1559681316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 msgr2=0x7f0ac4109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.489+0000 7f0acb200640 1 --2- 172.21.15.99:0/1559681316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4109660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009ca0 tx=0x7f0ac002c100 comp rx=0 tx=0).stop 2024-03-23T23:43:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 -- 172.21.15.99:0/1559681316 shutdown_connections 2024-03-23T23:43:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 --2- 172.21.15.99:0/1559681316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4109660 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 --2- 172.21.15.99:0/1559681316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4104940 0x7f0ac4106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 --2- 172.21.15.99:0/1559681316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4102010 0x7f0ac4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 -- 172.21.15.99:0/1559681316 >> 172.21.15.99:0/1559681316 conn(0x7f0ac40fb980 msgr2=0x7f0ac40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:48.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 -- 172.21.15.99:0/1559681316 shutdown_connections 2024-03-23T23:43:48.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.490+0000 7f0acb200640 1 -- 172.21.15.99:0/1559681316 wait complete. 2024-03-23T23:43:48.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 Processor -- start 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 -- start start 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4102010 0x7f0ac4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4104940 0x7f0ac40fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac410d1a0 con 0x7f0ac4102010 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac410d020 con 0x7f0ac4104940 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0acb200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac410d320 con 0x7f0ac4107270 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0ac9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4102010 0x7f0ac4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0aca800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0ac9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4104940 0x7f0ac40fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0aca800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57548/0 (socket says 172.21.15.99:57548) 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.491+0000 7f0aca800640 1 -- 172.21.15.99:0/2379963599 learned_addr learned my addr 172.21.15.99:0/2379963599 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 -- 172.21.15.99:0/2379963599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4104940 msgr2=0x7f0ac40fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4104940 0x7f0ac40fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 -- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4102010 msgr2=0x7f0ac4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4102010 0x7f0ac4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 -- 172.21.15.99:0/2379963599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4100d90 con 0x7f0ac4107270 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0aca800640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ac002c640 tx=0x7f0ac0035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0039040 con 0x7f0ac4107270 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ac0044070 con 0x7f0ac4107270 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac41a1880 con 0x7f0ac4107270 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.492+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00043c0 con 0x7f0ac4107270 2024-03-23T23:43:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.493+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41a1e20 con 0x7f0ac4107270 2024-03-23T23:43:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.493+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac40fd470 con 0x7f0ac4107270 2024-03-23T23:43:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.497+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0ac0004560 con 0x7f0ac4107270 2024-03-23T23:43:48.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.498+0000 7f0abea00640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 0x7f0aac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.498+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0ac00f0fb0 con 0x7f0ac4107270 2024-03-23T23:43:48.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.498+0000 7f0ac9e00640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 0x7f0aac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.499+0000 7f0ac9e00640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 0x7f0aac0a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009bd0 tx=0x7f0ab8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:48.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.502+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ac00be030 con 0x7f0ac4107270 2024-03-23T23:43:48.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.703+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac40622a0 con 0x7f0ac4107270 2024-03-23T23:43:48.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.705+0000 7f0abea00640 1 -- 172.21.15.99:0/2379963599 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f0ac00098b0 con 0x7f0ac4107270 2024-03-23T23:43:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:48.706 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 msgr2=0x7f0aac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 0x7f0aac0a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009bd0 tx=0x7f0ab8005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 msgr2=0x7f0ac4100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ac002c640 tx=0x7f0ac0035d10 comp rx=0 tx=0).stop 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 shutdown_connections 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f0aac0a2880 0x7f0aac0a4d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ac4107270 0x7f0ac4100500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ac4104940 0x7f0ac40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 --2- 172.21.15.99:0/2379963599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ac4102010 0x7f0ac4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 >> 172.21.15.99:0/2379963599 conn(0x7f0ac40fb980 msgr2=0x7f0ac4102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:48.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 shutdown_connections 2024-03-23T23:43:48.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.708+0000 7f0acb200640 1 -- 172.21.15.99:0/2379963599 wait complete. 2024-03-23T23:43:48.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:43:48.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 -- 172.21.15.99:0/2997684177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 msgr2=0x7f7688102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 --2- 172.21.15.99:0/2997684177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688102c10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 -- 172.21.15.99:0/2997684177 shutdown_connections 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 --2- 172.21.15.99:0/2997684177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7688103150 0x7f76881099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 --2- 172.21.15.99:0/2997684177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688102c10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 --2- 172.21.15.99:0/2997684177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76881015a0 0x7f76881019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 -- 172.21.15.99:0/2997684177 >> 172.21.15.99:0/2997684177 conn(0x7f76880fcb10 msgr2=0x7f76880fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:48.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 -- 172.21.15.99:0/2997684177 shutdown_connections 2024-03-23T23:43:48.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.935+0000 7f7687200640 1 -- 172.21.15.99:0/2997684177 wait complete. 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 Processor -- start 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 -- start start 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76881015a0 0x7f7688075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768810d270 con 0x7f76881015a0 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768810d0f0 con 0x7f7688102790 2024-03-23T23:43:48.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.937+0000 7f7687200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768810d3f0 con 0x7f7688103150 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76881015a0 0x7f7688075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7686800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7686800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34118/0 (socket says 172.21.15.99:34118) 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:46964/0 (socket says 172.21.15.99:46964) 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7686800640 1 -- 172.21.15.99:0/288239660 learned_addr learned my addr 172.21.15.99:0/288239660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 -- 172.21.15.99:0/288239660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 msgr2=0x7f7688073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 -- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76881015a0 msgr2=0x7f7688075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76881015a0 0x7f7688075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 -- 172.21.15.99:0/288239660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688074860 con 0x7f7688102790 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7686800640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.938+0000 7f7685400640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f767402c170 tx=0x7f767402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.939+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674043020 con 0x7f7688102790 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.939+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76740054e0 con 0x7f7688102790 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.939+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76881a1a30 con 0x7f7688102790 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.939+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674003c10 con 0x7f7688102790 2024-03-23T23:43:48.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.939+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76881a1fa0 con 0x7f7688102790 2024-03-23T23:43:48.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.940+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7688108860 con 0x7f7688102790 2024-03-23T23:43:48.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.942+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7674046020 con 0x7f7688102790 2024-03-23T23:43:48.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.943+0000 7f767ea00640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 0x7f766c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:48.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.943+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f76740f0ff0 con 0x7f7688102790 2024-03-23T23:43:48.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.943+0000 7f7685e00640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 0x7f766c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:48.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.944+0000 7f7685e00640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 0x7f766c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76881025f0 tx=0x7f7678005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:48.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:48.946+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f767404d020 con 0x7f7688102790 2024-03-23T23:43:49.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:49 smithi099 ceph-mon[31715]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:49 smithi099 ceph-mon[31715]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:49 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2379963599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.159+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f768802d2c0 con 0x7f7688102790 2024-03-23T23:43:49.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.159+0000 7f767ea00640 1 -- 172.21.15.99:0/288239660 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76740be030 con 0x7f7688102790 2024-03-23T23:43:49.159 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:43:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 msgr2=0x7f766c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 0x7f766c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76881025f0 tx=0x7f7678005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:49.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 msgr2=0x7f7688073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f767402c170 tx=0x7f767402c650 comp rx=0 tx=0).stop 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 shutdown_connections 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f766c0a2880 0x7f766c0a4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7688103150 0x7f7688073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688102790 0x7f7688073a90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 --2- 172.21.15.99:0/288239660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76881015a0 0x7f7688075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 >> 172.21.15.99:0/288239660 conn(0x7f76880fcb10 msgr2=0x7f7688105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 shutdown_connections 2024-03-23T23:43:49.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.162+0000 7f7687200640 1 -- 172.21.15.99:0/288239660 wait complete. 2024-03-23T23:43:49.185 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:43:49.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 -- 172.21.15.99:0/3412401044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4109870 msgr2=0x7fbae4110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 --2- 172.21.15.99:0/3412401044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4109870 0x7fbae4110100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009900 tx=0x7fbadc02bc40 comp rx=0 tx=0).stop 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 -- 172.21.15.99:0/3412401044 shutdown_connections 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 --2- 172.21.15.99:0/3412401044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4109870 0x7fbae4110100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 --2- 172.21.15.99:0/3412401044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4108eb0 0x7fbae4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 --2- 172.21.15.99:0/3412401044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4107cc0 0x7fbae41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 -- 172.21.15.99:0/3412401044 >> 172.21.15.99:0/3412401044 conn(0x7fbae4076c60 msgr2=0x7fbae40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 -- 172.21.15.99:0/3412401044 shutdown_connections 2024-03-23T23:43:49.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.388+0000 7fbae8c00640 1 -- 172.21.15.99:0/3412401044 wait complete. 2024-03-23T23:43:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.389+0000 7fbae8c00640 1 Processor -- start 2024-03-23T23:43:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 -- start start 2024-03-23T23:43:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 0x7fbae419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 0x7fbae419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4109870 0x7fbae41a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4113990 con 0x7fbae4109870 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4113810 con 0x7fbae4107cc0 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae8c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4113b10 con 0x7fbae4108eb0 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 0x7fbae419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbad9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 0x7fbae419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 0x7fbae419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:46980/0 (socket says 172.21.15.99:46980) 2024-03-23T23:43:49.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae1e00640 1 -- 172.21.15.99:0/1716005033 learned_addr learned my addr 172.21.15.99:0/1716005033 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.390+0000 7fbae2800640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4109870 0x7fbae41a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 -- 172.21.15.99:0/1716005033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 msgr2=0x7fbae419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 0x7fbae419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 -- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4109870 msgr2=0x7fbae41a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4109870 0x7fbae41a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 -- 172.21.15.99:0/1716005033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae41a5bc0 con 0x7fbae4108eb0 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbad9400640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 0x7fbae419e4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e9f0 tx=0x7fbacc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:49.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00ccc0 con 0x7fbae4108eb0 2024-03-23T23:43:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbacc005020 con 0x7fbae4108eb0 2024-03-23T23:43:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0049e0 con 0x7fbae4108eb0 2024-03-23T23:43:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41a5eb0 con 0x7fbae4108eb0 2024-03-23T23:43:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.391+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41a6370 con 0x7fbae4108eb0 2024-03-23T23:43:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.393+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae410ef80 con 0x7fbae4108eb0 2024-03-23T23:43:49.397 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:49 smithi149 ceph-mon[36083]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.397 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:49 smithi149 ceph-mon[36083]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.397 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:49 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2379963599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:49.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.394+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbacc005560 con 0x7fbae4108eb0 2024-03-23T23:43:49.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.396+0000 7fbadb400640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 0x7fbac80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.396+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbacc0cea20 con 0x7fbae4108eb0 2024-03-23T23:43:49.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.396+0000 7fbae1e00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 0x7fbac80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.397+0000 7fbae1e00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 0x7fbac80a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbad4011440 tx=0x7fbad4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:49.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.400+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbacc0051d0 con 0x7fbae4108eb0 2024-03-23T23:43:49.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:49 smithi174 ceph-mon[37002]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:49 smithi174 ceph-mon[37002]: from='client.25351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:49.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:49 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2379963599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:49.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.598+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbae4062340 con 0x7fbae4108eb0 2024-03-23T23:43:49.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.598+0000 7fbadb400640 1 -- 172.21.15.99:0/1716005033 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbacc09baa0 con 0x7fbae4108eb0 2024-03-23T23:43:49.599 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 msgr2=0x7fbac80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 0x7fbac80a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbad4011440 tx=0x7fbad4009500 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 msgr2=0x7fbae419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 0x7fbae419e4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e9f0 tx=0x7fbacc00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 shutdown_connections 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbac80a2880 0x7fbac80a4d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae4109870 0x7fbae41a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbae4108eb0 0x7fbae419e4e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 --2- 172.21.15.99:0/1716005033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbae4107cc0 0x7fbae419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.601+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 >> 172.21.15.99:0/1716005033 conn(0x7fbae4076c60 msgr2=0x7fbae410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.602+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 shutdown_connections 2024-03-23T23:43:49.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.602+0000 7fbae8c00640 1 -- 172.21.15.99:0/1716005033 wait complete. 2024-03-23T23:43:49.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.867+0000 7f9f60800640 1 -- 172.21.15.99:0/2469978050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58105470 msgr2=0x7f9f5810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.867+0000 7f9f60800640 1 --2- 172.21.15.99:0/2469978050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58105470 0x7f9f5810bd00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009ca0 tx=0x7f9f5402c100 comp rx=0 tx=0).stop 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.867+0000 7f9f60800640 1 -- 172.21.15.99:0/2469978050 shutdown_connections 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 --2- 172.21.15.99:0/2469978050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58105470 0x7f9f5810bd00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 --2- 172.21.15.99:0/2469978050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58104ab0 0x7f9f58104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 --2- 172.21.15.99:0/2469978050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 -- 172.21.15.99:0/2469978050 >> 172.21.15.99:0/2469978050 conn(0x7f9f580fee50 msgr2=0x7f9f58101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:49.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 -- 172.21.15.99:0/2469978050 shutdown_connections 2024-03-23T23:43:49.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.868+0000 7f9f60800640 1 -- 172.21.15.99:0/2469978050 wait complete. 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 Processor -- start 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 -- start start 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 0x7f9f581a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5810f540 con 0x7f9f581038c0 2024-03-23T23:43:49.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5810f3c0 con 0x7f9f58104ab0 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f60800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5810f6c0 con 0x7f9f58105470 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:46992/0 (socket says 172.21.15.99:46992) 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 0x7f9f581a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5d400640 1 -- 172.21.15.99:0/3712691435 learned_addr learned my addr 172.21.15.99:0/3712691435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.869+0000 7f9f5de00640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57892/0 (socket says 172.21.15.99:57892) 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 -- 172.21.15.99:0/3712691435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 msgr2=0x7f9f5819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 -- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 msgr2=0x7f9f58199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 -- 172.21.15.99:0/3712691435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f581a1770 con 0x7f9f58105470 2024-03-23T23:43:49.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5d400640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f5e800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 0x7f9f581a1100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f54037040 tx=0x7f9f54035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54039040 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f54044070 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f581a1a00 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54004350 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.870+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581a1ec0 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.872+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5810abe0 con 0x7f9f58105470 2024-03-23T23:43:49.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.873+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9f540044f0 con 0x7f9f58105470 2024-03-23T23:43:49.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.874+0000 7f9f52a00640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 0x7f9f400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:49.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.875+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9f540f0ff0 con 0x7f9f58105470 2024-03-23T23:43:49.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.875+0000 7f9f5de00640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 0x7f9f400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:49.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.875+0000 7f9f5de00640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 0x7f9f400a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9f58104910 tx=0x7f9f4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:49.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:49.878+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f540be010 con 0x7f9f58105470 2024-03-23T23:43:50.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.112+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f580622a0 con 0x7f9f58105470 2024-03-23T23:43:50.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.112+0000 7f9f52a00640 1 -- 172.21.15.99:0/3712691435 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f540098b0 con 0x7f9f58105470 2024-03-23T23:43:50.113 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.115+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 msgr2=0x7f9f400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.115+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 0x7f9f400a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9f58104910 tx=0x7f9f4c005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 msgr2=0x7f9f581a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 0x7f9f581a1100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f54037040 tx=0x7f9f54035d10 comp rx=0 tx=0).stop 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 shutdown_connections 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9f400a2800 0x7f9f400a4cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58105470 0x7f9f581a1100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58104ab0 0x7f9f5819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 --2- 172.21.15.99:0/3712691435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f581038c0 0x7f9f58199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 >> 172.21.15.99:0/3712691435 conn(0x7f9f580fee50 msgr2=0x7f9f58107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:50.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 shutdown_connections 2024-03-23T23:43:50.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.116+0000 7f9f60800640 1 -- 172.21.15.99:0/3712691435 wait complete. 2024-03-23T23:43:50.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:43:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 -- 172.21.15.99:0/2291093910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 msgr2=0x7fd910109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 --2- 172.21.15.99:0/2291093910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd910109b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd908009ca0 tx=0x7fd90802c0f0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 -- 172.21.15.99:0/2291093910 shutdown_connections 2024-03-23T23:43:50.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 --2- 172.21.15.99:0/2291093910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd910109b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 --2- 172.21.15.99:0/2291093910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd910102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 --2- 172.21.15.99:0/2291093910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 -- 172.21.15.99:0/2291093910 >> 172.21.15.99:0/2291093910 conn(0x7fd9100fcc50 msgr2=0x7fd9100ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 -- 172.21.15.99:0/2291093910 shutdown_connections 2024-03-23T23:43:50.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.349+0000 7fd916400640 1 -- 172.21.15.99:0/2291093910 wait complete. 2024-03-23T23:43:50.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.351+0000 7fd916400640 1 Processor -- start 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.351+0000 7fd916400640 1 -- start start 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.351+0000 7fd916400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.351+0000 7fd916400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd91019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.351+0000 7fd916400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd9101a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd916400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91010d340 con 0x7fd9101016c0 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd916400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91010d1c0 con 0x7fd910103270 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd916400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91010d4c0 con 0x7fd9101028b0 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd91019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57916/0 (socket says 172.21.15.99:57916) 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd9101a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 -- 172.21.15.99:0/3187388331 learned_addr learned my addr 172.21.15.99:0/3187388331 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 -- 172.21.15.99:0/3187388331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 msgr2=0x7fd91019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd91019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 -- 172.21.15.99:0/3187388331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 msgr2=0x7fd9101a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd9101a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90f400640 1 -- 172.21.15.99:0/3187388331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9101a17a0 con 0x7fd9101016c0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90fe00640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd9101a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.352+0000 7fd90ea00640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd91019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd90f400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e9c0 tx=0x7fd90000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000cd80 con 0x7fd9101016c0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9000035f0 con 0x7fd9101016c0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9101a1a90 con 0x7fd9101016c0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900010690 con 0x7fd9101016c0 2024-03-23T23:43:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.353+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101a1fe0 con 0x7fd9101016c0 2024-03-23T23:43:50.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.354+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9101a2270 con 0x7fd9101016c0 2024-03-23T23:43:50.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.356+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd9000037a0 con 0x7fd9101016c0 2024-03-23T23:43:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.358+0000 7fd907e00640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 0x7fd8f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.359+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd9000cee30 con 0x7fd9101016c0 2024-03-23T23:43:50.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.359+0000 7fd90ea00640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 0x7fd8f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.359+0000 7fd90ea00640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 0x7fd8f40a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91019b090 tx=0x7fd8fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:50.360 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:50 smithi099 ceph-mon[31715]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:50 smithi099 ceph-mon[31715]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:43:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:50 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/288239660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:50 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1716005033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.365+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd90009beb0 con 0x7fd9101016c0 2024-03-23T23:43:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:50 smithi149 ceph-mon[36083]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:50 smithi149 ceph-mon[36083]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:43:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:50 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/288239660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:50 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1716005033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:50 smithi174 ceph-mon[37002]: from='client.25357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T23:43:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:50 smithi174 ceph-mon[37002]: pgmap v398: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:43:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:50 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/288239660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:50 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1716005033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:43:50.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.613+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9101a29e0 con 0x7fd9101016c0 2024-03-23T23:43:50.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.614+0000 7fd907e00640 1 -- 172.21.15.99:0/3187388331 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fd90009b7b0 con 0x7fd9101016c0 2024-03-23T23:43:50.615 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:50.615 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:43:50.615 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:43:50.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 msgr2=0x7fd8f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 0x7fd8f40a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91019b090 tx=0x7fd8fc005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:50.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 msgr2=0x7fd910199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd90000e9c0 tx=0x7fd90000ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:50.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 shutdown_connections 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8f40a2800 0x7fd8f40a4cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910103270 0x7fd9101a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9101028b0 0x7fd91019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 --2- 172.21.15.99:0/3187388331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9101016c0 0x7fd910199b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 >> 172.21.15.99:0/3187388331 conn(0x7fd9100fcc50 msgr2=0x7fd9100738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 shutdown_connections 2024-03-23T23:43:50.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.617+0000 7fd916400640 1 -- 172.21.15.99:0/3187388331 wait complete. 2024-03-23T23:43:50.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.847+0000 7f897c000640 1 -- 172.21.15.99:0/1504906749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 msgr2=0x7f897410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.847+0000 7f897c000640 1 --2- 172.21.15.99:0/1504906749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f897410bd20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f896c009ca0 tx=0x7f896c02c100 comp rx=0 tx=0).stop 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 -- 172.21.15.99:0/1504906749 shutdown_connections 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 --2- 172.21.15.99:0/1504906749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f897410bd20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 --2- 172.21.15.99:0/1504906749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f8974104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 --2- 172.21.15.99:0/1504906749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 -- 172.21.15.99:0/1504906749 >> 172.21.15.99:0/1504906749 conn(0x7f89740fee90 msgr2=0x7f89741012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:50.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 -- 172.21.15.99:0/1504906749 shutdown_connections 2024-03-23T23:43:50.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.848+0000 7f897c000640 1 -- 172.21.15.99:0/1504906749 wait complete. 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 Processor -- start 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 -- start start 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f89741a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897410f560 con 0x7f89741038e0 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897410f3e0 con 0x7f8974105490 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897c000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897410f6e0 con 0x7f8974104ad0 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f8979600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f8978c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f8978c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34192/0 (socket says 172.21.15.99:34192) 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f8978c00640 1 -- 172.21.15.99:0/2214174476 learned_addr learned my addr 172.21.15.99:0/2214174476 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f8979600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57940/0 (socket says 172.21.15.99:57940) 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.849+0000 7f897a000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f89741a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 -- 172.21.15.99:0/2214174476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 msgr2=0x7f89741a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f89741a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 -- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 msgr2=0x7f8974199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 -- 172.21.15.99:0/2214174476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89741a1790 con 0x7f8974104ad0 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f897a000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f89741a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8978c00640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f896000e9f0 tx=0x7f896000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896000cd80 con 0x7f8974104ad0 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8960005500 con 0x7f8974104ad0 2024-03-23T23:43:50.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89741a1a80 con 0x7f8974104ad0 2024-03-23T23:43:50.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960010690 con 0x7f8974104ad0 2024-03-23T23:43:50.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.850+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89741a1f80 con 0x7f8974104ad0 2024-03-23T23:43:50.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.852+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897410ac00 con 0x7f8974104ad0 2024-03-23T23:43:50.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.854+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f89600046c0 con 0x7f8974104ad0 2024-03-23T23:43:50.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.855+0000 7f8972000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 0x7f895c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:50.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.855+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89600ceec0 con 0x7f8974104ad0 2024-03-23T23:43:50.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.856+0000 7f8979600640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 0x7f895c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:50.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.856+0000 7f8979600640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 0x7f895c0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8974104930 tx=0x7f8968005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:50.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:50.859+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f896009bee0 con 0x7f8974104ad0 2024-03-23T23:43:51.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.068+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89740622a0 con 0x7f8974104ad0 2024-03-23T23:43:51.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.070+0000 7f8972000640 1 -- 172.21.15.99:0/2214174476 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f896009b7e0 con 0x7f8974104ad0 2024-03-23T23:43:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:43:51.071 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:43:51.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 msgr2=0x7f895c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 0x7f895c0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8974104930 tx=0x7f8968005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 msgr2=0x7f897419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f896000e9f0 tx=0x7f896000edf0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 shutdown_connections 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f895c0a2800 0x7f895c0a4cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8974105490 0x7f89741a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8974104ad0 0x7f897419a0b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 --2- 172.21.15.99:0/2214174476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89741038e0 0x7f8974199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 >> 172.21.15.99:0/2214174476 conn(0x7f89740fee90 msgr2=0x7f8974107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 shutdown_connections 2024-03-23T23:43:51.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.073+0000 7f897c000640 1 -- 172.21.15.99:0/2214174476 wait complete. 2024-03-23T23:43:51.093 DEBUG:tasks.fwd_scrub:Yielding 2024-03-23T23:43:51.094 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-23T23:43:51.107 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:43:51.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-23T23:43:51.108 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-23T23:43:51.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T23:43:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:43:51.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T23:43:51.115 INFO:teuthology.run_tasks:Running task workunit... 2024-03-23T23:43:51.146 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:43:51.146 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-23T23:43:51.146 INFO:tasks.workunit:timeout=3h 2024-03-23T23:43:51.146 INFO:tasks.workunit:cleanup=True 2024-03-23T23:43:51.147 DEBUG:teuthology.orchestra.run.smithi099:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:51.184 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:51 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3712691435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:51.184 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:51 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3187388331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 76h/118d Inode: 1099511628283 Links: 2 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-23 23:41:18.375722625 +0000 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-23 23:41:18.375722625 +0000 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-23 23:43:40.949079047 +0000 2024-03-23T23:43:51.193 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-23 23:41:18.375722625 +0000 2024-03-23T23:43:51.194 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-23T23:43:51.194 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-23T23:43:51.320 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-23T23:43:51.323 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:51 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3712691435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:51.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:51 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3187388331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:51.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 -- 172.21.15.99:0/2277182355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54102000 msgr2=0x7f7f541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 --2- 172.21.15.99:0/2277182355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54102000 0x7f7f541043f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802bc30 comp rx=0 tx=0).stop 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 -- 172.21.15.99:0/2277182355 shutdown_connections 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 --2- 172.21.15.99:0/2277182355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54107260 0x7f7f541096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 --2- 172.21.15.99:0/2277182355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54104930 0x7f7f54106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 --2- 172.21.15.99:0/2277182355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54102000 0x7f7f541043f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.332+0000 7f7f5a000640 1 -- 172.21.15.99:0/2277182355 >> 172.21.15.99:0/2277182355 conn(0x7f7f540fba10 msgr2=0x7f7f540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.333+0000 7f7f5a000640 1 -- 172.21.15.99:0/2277182355 shutdown_connections 2024-03-23T23:43:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.333+0000 7f7f5a000640 1 -- 172.21.15.99:0/2277182355 wait complete. 2024-03-23T23:43:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.334+0000 7f7f5a000640 1 Processor -- start 2024-03-23T23:43:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.334+0000 7f7f5a000640 1 -- start start 2024-03-23T23:43:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54107260 0x7f7f541a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5410d260 con 0x7f7f54107260 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5410d0e0 con 0x7f7f54102000 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f5a000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5410d3e0 con 0x7f7f54104930 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47050/0 (socket says 172.21.15.99:47050) 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34218/0 (socket says 172.21.15.99:34218) 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f59600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54107260 0x7f7f541a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 -- 172.21.15.99:0/3725800357 learned_addr learned my addr 172.21.15.99:0/3725800357 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 -- 172.21.15.99:0/3725800357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 msgr2=0x7f7f54199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 -- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54107260 msgr2=0x7f7f541a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54107260 0x7f7f541a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.335+0000 7f7f58c00640 1 -- 172.21.15.99:0/3725800357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f541a16d0 con 0x7f7f54102000 2024-03-23T23:43:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f4802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f51600640 1 -- 172.21.15.99:0/3725800357 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4804e070 con 0x7f7f54102000 2024-03-23T23:43:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f51600640 1 -- 172.21.15.99:0/3725800357 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f48040be0 con 0x7f7f54102000 2024-03-23T23:43:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f54192940 con 0x7f7f54102000 2024-03-23T23:43:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f51600640 1 -- 172.21.15.99:0/3725800357 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480055d0 con 0x7f7f54102000 2024-03-23T23:43:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.336+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f54192e30 con 0x7f7f54102000 2024-03-23T23:43:51.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.339+0000 7f7f51600640 1 -- 172.21.15.99:0/3725800357 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7f48051070 con 0x7f7f54102000 2024-03-23T23:43:51.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f51600640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f51600640 1 -- 172.21.15.99:0/3725800357 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7f480f7ff0 con 0x7f7f54102000 2024-03-23T23:43:51.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f5e40f740 1 Processor -- start 2024-03-23T23:43:51.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f5e40f740 1 -- start start 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.341+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d02b0 0x55bad95d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad9535eb0 con 0x55bad95d02b0 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad94dea60 con 0x55bad95d5e10 2024-03-23T23:43:51.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad9531f50 con 0x55bad95d0ca0 2024-03-23T23:43:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f59600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f59600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47054/0 (socket says 172.21.15.99:47054) 2024-03-23T23:43:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f59600640 1 -- 172.21.15.99:0/501622518 learned_addr learned my addr 172.21.15.99:0/501622518 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f58c00640 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d02b0 0x55bad95d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f53e00640 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.342+0000 7f7f59600640 1 -- 172.21.15.99:0/501622518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 msgr2=0x55bad95d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f59600640 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f5419afc0 tx=0x7f7f40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f59600640 1 -- 172.21.15.99:0/501622518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d02b0 msgr2=0x55bad95d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f59600640 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d02b0 0x55bad95d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f59600640 1 -- 172.21.15.99:0/501622518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad93127e0 con 0x55bad95d5e10 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f59600640 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ccf2dc6185b55d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f45c00640 1 -- 172.21.15.99:0/501622518 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004d20 con 0x55bad95d5e10 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f45c00640 1 -- 172.21.15.99:0/501622518 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c004050 con 0x55bad95d5e10 2024-03-23T23:43:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.343+0000 7f7f45c00640 1 -- 172.21.15.99:0/501622518 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c005260 con 0x55bad95d5e10 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 -- 172.21.15.99:0/501622518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 msgr2=0x55bad95d82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009fd0 tx=0x7f7f4c0118d0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 -- 172.21.15.99:0/501622518 shutdown_connections 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95d82a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/501622518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d02b0 0x55bad95d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.344+0000 7f7f5e40f740 1 -- 172.21.15.99:0/501622518 >> 172.21.15.99:0/501622518 conn(0x55bad95cb820 msgr2=0x55bad95cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.345+0000 7f7f5e40f740 1 -- 172.21.15.99:0/501622518 shutdown_connections 2024-03-23T23:43:51.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.345+0000 7f7f5e40f740 1 -- 172.21.15.99:0/501622518 wait complete. 2024-03-23T23:43:51.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.345+0000 7f7f5e40f740 1 -- start start 2024-03-23T23:43:51.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.345+0000 7f7f5e40f740 1 Processor -- start 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 0x55bad95daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad95d8dc0 con 0x55bad95d5e10 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad94f1eb0 con 0x55bad95d0ca0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad95d8f40 con 0x55bad95d02b0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f5e40f740 10 client.0 _ll_register_callbacks cb 0x55bad94e6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f59600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f59600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57980/0 (socket says 172.21.15.99:57980) 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f59600640 1 -- 172.21.15.99:0/3913709949 learned_addr learned my addr 172.21.15.99:0/3913709949 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 0x55bad95daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 -- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 msgr2=0x55bad95daa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 0x55bad95daa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 -- 172.21.15.99:0/3913709949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 msgr2=0x55bad95e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.346+0000 7f7f58c00640 1 -- 172.21.15.99:0/3913709949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad9372f70 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 0x55bad95daa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f58c00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95da4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c4700 tx=0x7f7f48005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93e486c1b1069d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480c8aa0 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7f480c2d00 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.347+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f4c004050 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f480e2050 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 10 client.25405 fetch_fsmap learned FSMap version 27 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bad920a170 con 0x55bad95d02b0 2024-03-23T23:43:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c004050 con 0x55bad95d02b0 2024-03-23T23:43:51.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7f480c3e50 con 0x55bad95d02b0 2024-03-23T23:43:51.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 10 client.25405 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:43:51.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 20 client.25405 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad95e6310 0x55bad95e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 4 client.25405 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.348+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bad95433f0 con 0x55bad95e6310 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.349+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad95e6310 0x55bad95e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.349+0000 7f7f53e00640 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad95e6310 0x55bad95e87b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d4eed46dc6305 server_cookie=d270d23f9036ad34 in_seq=0 out_seq=0 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.349+0000 7f7f3a000640 10 client.25405 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.350+0000 7f7f3a000640 1 -- 172.21.15.99:0/3913709949 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bad95433f0 con 0x55bad95e6310 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.350+0000 7f7f3a000640 10 client.25405 handle_command_reply: tid=0 2024-03-23T23:43:51.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.350+0000 7f7f5e40f740 1 client.25405 shutdown 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 msgr2=0x55bad95da4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95da4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c4700 tx=0x7f7f48005e50 comp rx=0 tx=0).stop 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 shutdown_connections 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad95e6310 0x55bad95e87b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d0ca0 0x55bad95daa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/3913709949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95da4c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 >> 172.21.15.99:0/3913709949 conn(0x55bad95cb820 msgr2=0x55bad95caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.351+0000 7f7f3a000640 0 client.25405 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.352+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 shutdown_connections 2024-03-23T23:43:51.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.352+0000 7f7f5e40f740 1 -- 172.21.15.99:0/3913709949 wait complete. 2024-03-23T23:43:51.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.352+0000 7f7f5e40f740 20 client.25405 trim_cache size 0 max 16384 2024-03-23T23:43:51.367 INFO:tasks.workunit.client.0.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-23T23:43:51.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 Processor -- start 2024-03-23T23:43:51.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 -- start start 2024-03-23T23:43:51.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d02b0 0x55bad95d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a6640 con 0x55bad95d5e10 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a7fc0 con 0x55bad95d02b0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a6ca0 con 0x55bad95d0ca0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f53e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34242/0 (socket says 172.21.15.99:34242) 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f53e00640 1 -- 172.21.15.99:0/4181600531 learned_addr learned my addr 172.21.15.99:0/4181600531 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.377+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d02b0 0x55bad95d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f59600640 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 -- 172.21.15.99:0/4181600531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d02b0 msgr2=0x55bad95d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d02b0 0x55bad95d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 -- 172.21.15.99:0/4181600531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 msgr2=0x55bad95d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 -- 172.21.15.99:0/4181600531 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad95e8e30 con 0x55bad95d0ca0 2024-03-23T23:43:51.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f59600640 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:43:51.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f53e00640 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f40013010 tx=0x7f7f40014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7d59e3e13e975 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.378+0000 7f7f45200640 1 -- 172.21.15.99:0/4181600531 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400299d0 con 0x55bad95d0ca0 2024-03-23T23:43:51.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.379+0000 7f7f45200640 1 -- 172.21.15.99:0/4181600531 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f40029b70 con 0x55bad95d0ca0 2024-03-23T23:43:51.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.379+0000 7f7f45200640 1 -- 172.21.15.99:0/4181600531 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000f7e0 con 0x55bad95d0ca0 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- 172.21.15.99:0/4181600531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 msgr2=0x55bad95e19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f40013010 tx=0x7f7f40014000 comp rx=0 tx=0).stop 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- 172.21.15.99:0/4181600531 shutdown_connections 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d5e10 0x55bad95d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d0ca0 0x55bad95e19d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/4181600531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d02b0 0x55bad95d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- 172.21.15.99:0/4181600531 >> 172.21.15.99:0/4181600531 conn(0x55bad95cb820 msgr2=0x55bad95d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- 172.21.15.99:0/4181600531 shutdown_connections 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- 172.21.15.99:0/4181600531 wait complete. 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.380+0000 7f7f5e40f740 1 -- start start 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 Processor -- start 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 0x55bad95e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a7960 con 0x55bad95d0ca0 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a7300 con 0x55bad95d5e10 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541a47e0 con 0x55bad95d02b0 2024-03-23T23:43:51.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f5e40f740 10 client.0 _ll_register_callbacks cb 0x7f7f480c0ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.381+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f59600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34248/0 (socket says 172.21.15.99:34248) 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 -- 172.21.15.99:0/2588831234 learned_addr learned my addr 172.21.15.99:0/2588831234 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f53e00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 0x55bad95e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 -- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 msgr2=0x55bad95e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 -- 172.21.15.99:0/2588831234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 msgr2=0x55bad95e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 0x55bad95e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 -- 172.21.15.99:0/2588831234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad95cc590 con 0x55bad95d02b0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f59600640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f53e00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 0x55bad95e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f58c00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c1380 tx=0x7f7f48005330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547c8d2ea840f833 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480e2050 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.382+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7f480c2490 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f40029b70 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f480c4170 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 10 client.25411 fetch_fsmap learned FSMap version 27 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bad9643050 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f40029b70 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7f480c3e50 con 0x55bad95d02b0 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 10 client.25411 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 20 client.25411 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad96437f0 0x55bad9645c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 4 client.25411 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T23:43:51.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.383+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bad953d740 con 0x55bad96437f0 2024-03-23T23:43:51.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.384+0000 7f7f53e00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad96437f0 0x55bad9645c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.384+0000 7f7f53e00640 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad96437f0 0x55bad9645c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ddccbb79e137211 server_cookie=4ce2593b8d189329 in_seq=0 out_seq=0 2024-03-23T23:43:51.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.384+0000 7f7f3a000640 10 client.25411 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.385+0000 7f7f3a000640 1 -- 172.21.15.99:0/2588831234 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bad953d740 con 0x55bad96437f0 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.385+0000 7f7f3a000640 10 client.25411 handle_command_reply: tid=0 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 client.25411 shutdown 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 msgr2=0x55bad95e04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f480c1380 tx=0x7f7f48005330 comp rx=0 tx=0).stop 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 shutdown_connections 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bad96437f0 0x55bad9645c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bad95d5e10 0x55bad95e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad95d0ca0 0x55bad95e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 --2- 172.21.15.99:0/2588831234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad95d02b0 0x55bad95e04c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f3a000640 0 client.25411 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 >> 172.21.15.99:0/2588831234 conn(0x55bad95cb820 msgr2=0x55bad95d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 shutdown_connections 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 1 -- 172.21.15.99:0/2588831234 wait complete. 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.386+0000 7f7f5e40f740 20 client.25411 trim_cache size 0 max 16384 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "48d6a150-5051-4b76-ac68-14c4c54d4b17", 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-23T23:43:51.389 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 msgr2=0x7f7f3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f5419afc0 tx=0x7f7f40005f90 comp rx=0 tx=0).stop 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 msgr2=0x7f7f54199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f4802c650 comp rx=0 tx=0).stop 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 shutdown_connections 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f54107260 0x7f7f541a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f54104930 0x7f7f54199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 --2- 172.21.15.99:0/3725800357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f54102000 0x7f7f54199ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.387+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 >> 172.21.15.99:0/3725800357 conn(0x7f7f540fba10 msgr2=0x7f7f541016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.388+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 shutdown_connections 2024-03-23T23:43:51.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.388+0000 7f7f5a000640 1 -- 172.21.15.99:0/3725800357 wait complete. 2024-03-23T23:43:51.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:51 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3712691435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:43:51.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:51 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3187388331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:43:51.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:43:51.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.669+0000 7f081b200640 1 -- 172.21.15.99:0/3625595446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 msgr2=0x7f08141099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.669+0000 7f081b200640 1 --2- 172.21.15.99:0/3625595446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141099e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002c100 comp rx=0 tx=0).stop 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 -- 172.21.15.99:0/3625595446 shutdown_connections 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 --2- 172.21.15.99:0/3625595446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141099e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 --2- 172.21.15.99:0/3625595446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 --2- 172.21.15.99:0/3625595446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f08141019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 -- 172.21.15.99:0/3625595446 >> 172.21.15.99:0/3625595446 conn(0x7f08140fcb10 msgr2=0x7f08140fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 -- 172.21.15.99:0/3625595446 shutdown_connections 2024-03-23T23:43:51.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.670+0000 7f081b200640 1 -- 172.21.15.99:0/3625595446 wait complete. 2024-03-23T23:43:51.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.671+0000 7f081b200640 1 Processor -- start 2024-03-23T23:43:51.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.671+0000 7f081b200640 1 -- start start 2024-03-23T23:43:51.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081410d230 con 0x7f0814102790 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081410d0b0 con 0x7f08141015a0 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081410d3b0 con 0x7f0814103150 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47092/0 (socket says 172.21.15.99:47092) 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 -- 172.21.15.99:0/3891276136 learned_addr learned my addr 172.21.15.99:0/3891276136 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819400640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f081a800640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 -- 172.21.15.99:0/3891276136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 msgr2=0x7f08141a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.672+0000 7f0819e00640 1 -- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 msgr2=0x7f0814199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f0819e00640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f0819e00640 1 -- 172.21.15.99:0/3891276136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08141a1690 con 0x7f08141015a0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f0819400640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f0819e00640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f080ea00640 1 -- 172.21.15.99:0/3891276136 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800cd80 con 0x7f08141015a0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f080ea00640 1 -- 172.21.15.99:0/3891276136 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08080035f0 con 0x7f08141015a0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08141a1980 con 0x7f08141015a0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f080ea00640 1 -- 172.21.15.99:0/3891276136 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808010690 con 0x7f08141015a0 2024-03-23T23:43:51.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.673+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141a1ed0 con 0x7f08141015a0 2024-03-23T23:43:51.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.677+0000 7f081f42d740 1 Processor -- start 2024-03-23T23:43:51.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.677+0000 7f081f42d740 1 -- start start 2024-03-23T23:43:51.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83b85250 con 0x55dd83c25bb0 2024-03-23T23:43:51.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83b2da30 con 0x55dd83c20050 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081f42d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83b812f0 con 0x55dd83c20a40 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47096/0 (socket says 172.21.15.99:47096) 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 -- 172.21.15.99:0/3324666045 learned_addr learned my addr 172.21.15.99:0/3324666045 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f080ea00640 1 -- 172.21.15.99:0/3891276136 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f08080037a0 con 0x7f08141015a0 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f0819400640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f0819e00640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 -- 172.21.15.99:0/3324666045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 msgr2=0x55dd83c25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c25540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 -- 172.21.15.99:0/3324666045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 msgr2=0x55dd83c28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.678+0000 7f081a800640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f081a800640 1 -- 172.21.15.99:0/3324666045 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd83b18690 con 0x55dd83c20050 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f0819e00640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f081a800640 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f08100323c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12cd292c6b92c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f0803000640 1 -- 172.21.15.99:0/3324666045 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810043070 con 0x55dd83c20050 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f0803000640 1 -- 172.21.15.99:0/3324666045 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0810003970 con 0x55dd83c20050 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.679+0000 7f0803000640 1 -- 172.21.15.99:0/3324666045 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810039040 con 0x55dd83c20050 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 -- 172.21.15.99:0/3324666045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 msgr2=0x55dd83c20470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f08100323c0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 -- 172.21.15.99:0/3324666045 shutdown_connections 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 --2- 172.21.15.99:0/3324666045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c20470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 -- 172.21.15.99:0/3324666045 >> 172.21.15.99:0/3324666045 conn(0x55dd83be8200 msgr2=0x55dd83c1b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f080ea00640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 0x7f07fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.680+0000 7f081f42d740 1 -- 172.21.15.99:0/3324666045 shutdown_connections 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.681+0000 7f081f42d740 1 -- 172.21.15.99:0/3324666045 wait complete. 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.681+0000 7f080ea00640 1 -- 172.21.15.99:0/3891276136 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f08080cef40 con 0x7f08141015a0 2024-03-23T23:43:51.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.681+0000 7f081a800640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 0x7f07fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.681+0000 7f081f42d740 1 -- start start 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.681+0000 7f081f42d740 1 Processor -- start 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081a800640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 0x7f07fc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0810005e30 tx=0x7f081002c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83c28b00 con 0x55dd83c25bb0 2024-03-23T23:43:51.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83b40e80 con 0x55dd83c20a40 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd83c28c80 con 0x55dd83c20050 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081f42d740 10 client.0 _ll_register_callbacks cb 0x55dd83b53130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f0819e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f0819e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34270/0 (socket says 172.21.15.99:34270) 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f0819e00640 1 -- 172.21.15.99:0/3986365044 learned_addr learned my addr 172.21.15.99:0/3986365044 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f0819400640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.682+0000 7f081a800640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 -- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 msgr2=0x55dd83c2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 -- 172.21.15.99:0/3986365044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 msgr2=0x55dd83c315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 -- 172.21.15.99:0/3986365044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd839637d0 con 0x55dd83c20050 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f081a800640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:43:51.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819400640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f0819e00640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f080800c6a0 tx=0x7f080809b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d325a8d350cf63 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f07f7400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080809f9b0 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0808003b30 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.683+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0810003970 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08080a4000 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 10 client.25420 fetch_fsmap learned FSMap version 27 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd838993f0 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0810003970 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f080809cb90 con 0x55dd83c20050 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 10 client.25420 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 10 client.25420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 20 client.25420 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c35df0 0x55dd83c38270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 4 client.25420 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:43:51.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.684+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd83b92790 con 0x55dd83c35df0 2024-03-23T23:43:51.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.685+0000 7f0819400640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c35df0 0x55dd83c38270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.685+0000 7f0819400640 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c35df0 0x55dd83c38270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b7e3b518b0c884 server_cookie=94e18d79749a75e1 in_seq=0 out_seq=0 2024-03-23T23:43:51.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.685+0000 7f07f7400640 10 client.25420 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f07f7400640 1 -- 172.21.15.99:0/3986365044 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd83b92790 con 0x55dd83c35df0 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f07f7400640 10 client.25420 handle_command_reply: tid=0 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 client.25420 shutdown 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 msgr2=0x55dd83c2a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f080800c6a0 tx=0x7f080809b490 comp rx=0 tx=0).stop 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 shutdown_connections 2024-03-23T23:43:51.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c35df0 0x55dd83c38270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 --2- 172.21.15.99:0/3986365044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2a040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 >> 172.21.15.99:0/3986365044 conn(0x55dd83be8200 msgr2=0x55dd83c1abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f07f7400640 0 client.25420 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 shutdown_connections 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.686+0000 7f081f42d740 1 -- 172.21.15.99:0/3986365044 wait complete. 2024-03-23T23:43:51.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.687+0000 7f081f42d740 20 client.25420 trim_cache size 0 max 16384 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 Processor -- start 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 -- start start 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141aa9e0 con 0x55dd83c25bb0 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141ac360 con 0x55dd83c20050 2024-03-23T23:43:51.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081f42d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141ab040 con 0x55dd83c20a40 2024-03-23T23:43:51.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f0819400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f0819e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f0819400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34274/0 (socket says 172.21.15.99:34274) 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f0819400640 1 -- 172.21.15.99:0/2939803448 learned_addr learned my addr 172.21.15.99:0/2939803448 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f0819e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c2a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47118/0 (socket says 172.21.15.99:47118) 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.704+0000 7f081a800640 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 -- 172.21.15.99:0/2939803448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 msgr2=0x55dd83c2a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c2a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 -- 172.21.15.99:0/2939803448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 msgr2=0x55dd83c1f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c1f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 -- 172.21.15.99:0/2939803448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd838590f0 con 0x55dd83c20a40 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0819400640 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55dd83c1c910 tx=0x7f080401ea50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2059aba9bcc3fffa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0802600640 1 -- 172.21.15.99:0/2939803448 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804024070 con 0x55dd83c20a40 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0802600640 1 -- 172.21.15.99:0/2939803448 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0804003cd0 con 0x55dd83c20a40 2024-03-23T23:43:51.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.705+0000 7f0802600640 1 -- 172.21.15.99:0/2939803448 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804012cc0 con 0x55dd83c20a40 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- 172.21.15.99:0/2939803448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 msgr2=0x55dd83c31550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55dd83c1c910 tx=0x7f080401ea50 comp rx=0 tx=0).stop 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- 172.21.15.99:0/2939803448 shutdown_connections 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20a40 0x55dd83c31550 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 --2- 172.21.15.99:0/2939803448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20050 0x55dd83c2a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- 172.21.15.99:0/2939803448 >> 172.21.15.99:0/2939803448 conn(0x55dd83be8200 msgr2=0x55dd83c1ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- 172.21.15.99:0/2939803448 shutdown_connections 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- 172.21.15.99:0/2939803448 wait complete. 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.706+0000 7f081f42d740 1 -- start start 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 Processor -- start 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c30170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141abd00 con 0x55dd83c25bb0 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141ab6a0 con 0x55dd83c20a40 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141a8ba0 con 0x55dd83c20050 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081f42d740 10 client.0 _ll_register_callbacks cb 0x7f08080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f0819e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f0819e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:34278/0 (socket says 172.21.15.99:34278) 2024-03-23T23:43:51.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f0819e00640 1 -- 172.21.15.99:0/143856387 learned_addr learned my addr 172.21.15.99:0/143856387 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f0819400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.707+0000 7f081a800640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c30170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 -- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 msgr2=0x55dd83c2f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 -- 172.21.15.99:0/143856387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 msgr2=0x55dd83c30170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c30170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 -- 172.21.15.99:0/143856387 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd83c30760 con 0x55dd83c20050 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f0819e00640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f08080981e0 tx=0x7f0808000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b41c16446f914a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f07f7400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808064030 con 0x55dd83c20050 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f080809cb90 con 0x55dd83c20050 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.708+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0804003cd0 con 0x55dd83c20050 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0808080070 con 0x55dd83c20050 2024-03-23T23:43:51.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 10 client.25426 fetch_fsmap learned FSMap version 27 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd83c917a0 con 0x55dd83c20050 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0804003cd0 con 0x55dd83c20050 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f080809a720 con 0x55dd83c20050 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 10 client.25426 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 20 client.25426 populate_metadata read hostname 'smithi099' 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c920c0 0x55dd83c944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 4 client.25426 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd83b8cae0 con 0x55dd83c920c0 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.709+0000 7f0819400640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c920c0 0x55dd83c944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f0819400640 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c920c0 0x55dd83c944e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7bf3f049ab117d server_cookie=f31ab88ff22ed520 in_seq=0 out_seq=0 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f07f7400640 10 client.25426 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f07f7400640 1 -- 172.21.15.99:0/143856387 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd83b8cae0 con 0x55dd83c920c0 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f07f7400640 10 client.25426 handle_command_reply: tid=0 2024-03-23T23:43:51.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 client.25426 shutdown 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 msgr2=0x55dd83c2ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f08080981e0 tx=0x7f0808000bc0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 shutdown_connections 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55dd83c920c0 0x55dd83c944e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dd83c25bb0 0x55dd83c30170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd83c20a40 0x55dd83c2f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 --2- 172.21.15.99:0/143856387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd83c20050 0x55dd83c2ec20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 >> 172.21.15.99:0/143856387 conn(0x55dd83be8200 msgr2=0x55dd83c2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.710+0000 7f07f7400640 0 client.25426 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:43:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 shutdown_connections 2024-03-23T23:43:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081f42d740 1 -- 172.21.15.99:0/143856387 wait complete. 2024-03-23T23:43:51.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081f42d740 20 client.25426 trim_cache size 0 max 16384 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "48d6a150-5051-4b76-ac68-14c4c54d4b17": { 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "48d6a150-5051-4b76-ac68-14c4c54d4b17", 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:43:51.719 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 msgr2=0x7f07fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 0x7f07fc0a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0810005e30 tx=0x7f081002c100 comp rx=0 tx=0).stop 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 msgr2=0x7f0814199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.711+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).stop 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 shutdown_connections 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f07fc0a2800 0x7f07fc0a4cc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814103150 0x7f08141a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0814102790 0x7f0814199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 --2- 172.21.15.99:0/3891276136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08141015a0 0x7f0814199a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:43:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 >> 172.21.15.99:0/3891276136 conn(0x7f08140fcb10 msgr2=0x7f08140738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:43:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 shutdown_connections 2024-03-23T23:43:51.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:43:51.712+0000 7f081b200640 1 -- 172.21.15.99:0/3891276136 wait complete. 2024-03-23T23:43:51.732 INFO:tasks.cephfs.filesystem:scrub status for tag:48d6a150-5051-4b76-ac68-14c4c54d4b17 - {'path': '/', 'tag': '48d6a150-5051-4b76-ac68-14c4c54d4b17', 'options': 'recursive,force'} 2024-03-23T23:43:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:52 smithi099 ceph-mon[31715]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:52 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2214174476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:52.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:52 smithi149 ceph-mon[36083]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:52 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2214174476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:52.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:52 smithi174 ceph-mon[37002]: pgmap v399: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-23T23:43:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:52 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2214174476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:43:53.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:53 smithi099 ceph-mon[31715]: scrub queued for path: / 2024-03-23T23:43:53.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:53 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:53.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:53 smithi099 ceph-mon[31715]: scrub summary: active paths [/] 2024-03-23T23:43:53.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:53 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:53 smithi149 ceph-mon[36083]: scrub queued for path: / 2024-03-23T23:43:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:53 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:53 smithi149 ceph-mon[36083]: scrub summary: active paths [/] 2024-03-23T23:43:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:53 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:53.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:53 smithi174 ceph-mon[37002]: scrub queued for path: / 2024-03-23T23:43:53.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:53 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:53.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:53 smithi174 ceph-mon[37002]: scrub summary: active paths [/] 2024-03-23T23:43:53.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:53 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:43:54.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:54 smithi099 ceph-mon[31715]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:43:54.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:54 smithi149 ceph-mon[36083]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:43:54.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:54 smithi174 ceph-mon[37002]: pgmap v400: 97 pgs: 97 active+clean; 651 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:43:56.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:56 smithi099 ceph-mon[31715]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:43:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:56 smithi149 ceph-mon[36083]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:43:56.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:56 smithi174 ceph-mon[37002]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:43:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:43:58 smithi099 ceph-mon[31715]: pgmap v402: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T23:43:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:43:58 smithi149 ceph-mon[36083]: pgmap v402: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T23:43:58.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:43:58 smithi174 ceph-mon[37002]: pgmap v402: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-23T23:44:00.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:00 smithi099 ceph-mon[31715]: pgmap v403: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:00.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:00 smithi149 ceph-mon[36083]: pgmap v403: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:00.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:00 smithi174 ceph-mon[37002]: pgmap v403: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:02.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:02 smithi099 ceph-mon[31715]: pgmap v404: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:02.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:02 smithi149 ceph-mon[36083]: pgmap v404: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:02.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:02 smithi174 ceph-mon[37002]: pgmap v404: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-23T23:44:04.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:04 smithi099 ceph-mon[31715]: pgmap v405: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:44:04.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:04 smithi099 ceph-mon[31715]: scrub summary: idle 2024-03-23T23:44:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 -- 172.21.15.99:0/3623013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41016e0 msgr2=0x7fb9b4101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 --2- 172.21.15.99:0/3623013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41016e0 0x7fb9b4101b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009900 tx=0x7fb9a402bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 -- 172.21.15.99:0/3623013844 shutdown_connections 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 --2- 172.21.15.99:0/3623013844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b4103290 0x7fb9b4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 --2- 172.21.15.99:0/3623013844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b41028d0 0x7fb9b4102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 --2- 172.21.15.99:0/3623013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41016e0 0x7fb9b4101b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 -- 172.21.15.99:0/3623013844 >> 172.21.15.99:0/3623013844 conn(0x7fb9b40fcc30 msgr2=0x7fb9b40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.379+0000 7fb9b9800640 1 -- 172.21.15.99:0/3623013844 shutdown_connections 2024-03-23T23:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.380+0000 7fb9b9800640 1 -- 172.21.15.99:0/3623013844 wait complete. 2024-03-23T23:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.380+0000 7fb9b9800640 1 Processor -- start 2024-03-23T23:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 -- start start 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b4103290 0x7fb9b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b410d380 con 0x7fb9b4103290 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b410d200 con 0x7fb9b41016e0 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b9800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b410d500 con 0x7fb9b41028d0 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52454/0 (socket says 172.21.15.99:52454) 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b4103290 0x7fb9b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b2000640 1 -- 172.21.15.99:0/1767380541 learned_addr learned my addr 172.21.15.99:0/1767380541 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.381+0000 7fb9b2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37524/0 (socket says 172.21.15.99:37524) 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 -- 172.21.15.99:0/1767380541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 msgr2=0x7fb9b4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 -- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b4103290 msgr2=0x7fb9b419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b4103290 0x7fb9b419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 -- 172.21.15.99:0/1767380541 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b419f560 con 0x7fb9b41016e0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009230 tx=0x7fb9a402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9ab400640 1 -- 172.21.15.99:0/1767380541 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4043020 con 0x7fb9b41016e0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9ab400640 1 -- 172.21.15.99:0/1767380541 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40054e0 con 0x7fb9b41016e0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b419f7f0 con 0x7fb9b41016e0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.382+0000 7fb9ab400640 1 -- 172.21.15.99:0/1767380541 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4003b60 con 0x7fb9b41016e0 2024-03-23T23:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.383+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b419fd60 con 0x7fb9b41016e0 2024-03-23T23:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.385+0000 7fb9ab400640 1 -- 172.21.15.99:0/1767380541 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb9a4046020 con 0x7fb9b41016e0 2024-03-23T23:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 Processor -- start 2024-03-23T23:44:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 -- start start 2024-03-23T23:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901cbd0 0x564f990216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f98fdf3b0 con 0x564f99021d40 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f98f81830 con 0x564f9901c1e0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.386+0000 7fb9bc475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f98f2a5d0 con 0x564f9901cbd0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37526/0 (socket says 172.21.15.99:37526) 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 -- 172.21.15.99:0/3040742488 learned_addr learned my addr 172.21.15.99:0/3040742488 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901cbd0 0x564f990216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b2000640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 -- 172.21.15.99:0/3040742488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901cbd0 msgr2=0x564f990216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901cbd0 0x564f990216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 -- 172.21.15.99:0/3040742488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 msgr2=0x564f990241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b3400640 1 -- 172.21.15.99:0/3040742488 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f98f422c0 con 0x564f9901c1e0 2024-03-23T23:44:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.387+0000 7fb9b2000640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb9b3400640 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009fd0 tx=0x7fb9ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a880e080a82e5a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb9ab400640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 0x7fb9980a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb99fe00640 1 -- 172.21.15.99:0/3040742488 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004d20 con 0x564f9901c1e0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb99fe00640 1 -- 172.21.15.99:0/3040742488 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac004050 con 0x564f9901c1e0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb99fe00640 1 -- 172.21.15.99:0/3040742488 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac005260 con 0x564f9901c1e0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb9ab400640 1 -- 172.21.15.99:0/1767380541 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9a40f0a30 con 0x7fb9b41016e0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.388+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 0x7fb9980a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 -- 172.21.15.99:0/3040742488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 msgr2=0x564f9901c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009fd0 tx=0x7fb9ac0118d0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 -- 172.21.15.99:0/3040742488 shutdown_connections 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901cbd0 0x564f990216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 --2- 172.21.15.99:0/3040742488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f9901c600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 -- 172.21.15.99:0/3040742488 >> 172.21.15.99:0/3040742488 conn(0x564f99017750 msgr2=0x564f99017b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 -- 172.21.15.99:0/3040742488 shutdown_connections 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9b2a00640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 0x7fb9980a4de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c7b0 tx=0x7fb9a40dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.389+0000 7fb9bc475740 1 -- 172.21.15.99:0/3040742488 wait complete. 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.390+0000 7fb9bc475740 1 -- start start 2024-03-23T23:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.390+0000 7fb9bc475740 1 Processor -- start 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.390+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.390+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f99026930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f9902d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9bc475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f98f3da20 con 0x564f9901cbd0 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9bc475740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f98f7d8d0 con 0x564f9901c1e0 2024-03-23T23:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9bc475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f99024cf0 con 0x564f99021d40 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9bc475740 10 client.0 _ll_register_callbacks cb 0x564f98f7a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37540/0 (socket says 172.21.15.99:37540) 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 -- 172.21.15.99:0/1768957494 learned_addr learned my addr 172.21.15.99:0/1768957494 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f9902d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f99026930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 -- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 msgr2=0x564f990263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 -- 172.21.15.99:0/1768957494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 msgr2=0x564f99026930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f99026930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.391+0000 7fb9b3400640 1 -- 172.21.15.99:0/1768957494 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f98c96210 con 0x564f99021d40 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f9902d970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac011dd0 tx=0x7fb9ac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7cf0ae0b47f15a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb99cc00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac02d060 con 0x564f99021d40 2024-03-23T23:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9ac00f4e0 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9ac004050 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9ac00f6e0 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb9bc475740 10 client.25438 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f98d62620 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.392+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac004050 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9ac025040 con 0x564f99021d40 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 10 client.25438 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 20 client.25438 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f99032020 0x564f99034410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 4 client.25438 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f98f8ed70 con 0x564f99032020 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.393+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f99032020 0x564f99034410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.394+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f99032020 0x564f99034410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2db8114a680ed13b server_cookie=c03a4ba0e590538f in_seq=0 out_seq=0 2024-03-23T23:44:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.394+0000 7fb99cc00640 10 client.25438 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:04.397 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:04 smithi149 ceph-mon[36083]: pgmap v405: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:44:04.397 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:04 smithi149 ceph-mon[36083]: scrub summary: idle 2024-03-23T23:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb99cc00640 1 -- 172.21.15.99:0/1768957494 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f98f8ed70 con 0x564f99032020 2024-03-23T23:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb99cc00640 10 client.25438 handle_command_reply: tid=0 2024-03-23T23:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 client.25438 shutdown 2024-03-23T23:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 msgr2=0x564f9902d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f9902d970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac011dd0 tx=0x7fb9ac005fd0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 shutdown_connections 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f99032020 0x564f99034410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f9902d970 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f99026930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb99cc00640 0 client.25438 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1768957494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 >> 172.21.15.99:0/1768957494 conn(0x564f99017750 msgr2=0x564f99016950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 shutdown_connections 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 1 -- 172.21.15.99:0/1768957494 wait complete. 2024-03-23T23:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.395+0000 7fb9bc475740 20 client.25438 trim_cache size 0 max 16384 2024-03-23T23:44:04.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:04 smithi174 ceph-mon[37002]: pgmap v405: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-03-23T23:44:04.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:04 smithi174 ceph-mon[37002]: scrub summary: idle 2024-03-23T23:44:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.424+0000 7fb9bc475740 1 Processor -- start 2024-03-23T23:44:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.424+0000 7fb9bc475740 1 -- start start 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f990160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a8990 con 0x564f9901cbd0 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41aa310 con 0x564f9901c1e0 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9bc475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a8ff0 con 0x564f99021d40 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37548/0 (socket says 172.21.15.99:37548) 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39678/0 (socket says 172.21.15.99:39678) 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b3400640 1 -- 172.21.15.99:0/2815093176 learned_addr learned my addr 172.21.15.99:0/2815093176 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b3400640 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f990160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.425+0000 7fb9b2000640 1 -- 172.21.15.99:0/2815093176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 msgr2=0x564f990160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b2000640 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f990160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b2000640 1 -- 172.21.15.99:0/2815093176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 msgr2=0x564f990123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b2000640 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b2000640 1 -- 172.21.15.99:0/2815093176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f9901c5c0 con 0x564f9901cbd0 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b3400640 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb9b2000640 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x564f99034bd0 tx=0x7fb9a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4f8488ebcf194d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb99f400640 1 -- 172.21.15.99:0/2815093176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0024070 con 0x564f9901cbd0 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb99f400640 1 -- 172.21.15.99:0/2815093176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a0005500 con 0x564f9901cbd0 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.426+0000 7fb99f400640 1 -- 172.21.15.99:0/2815093176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0004b20 con 0x564f9901cbd0 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- 172.21.15.99:0/2815093176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 msgr2=0x564f9902d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x564f99034bd0 tx=0x7fb9a0005f90 comp rx=0 tx=0).stop 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- 172.21.15.99:0/2815093176 shutdown_connections 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f99021d40 0x564f990160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f9901cbd0 0x564f9902d930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 --2- 172.21.15.99:0/2815093176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901c1e0 0x564f990123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- 172.21.15.99:0/2815093176 >> 172.21.15.99:0/2815093176 conn(0x564f99017750 msgr2=0x564f9901def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- 172.21.15.99:0/2815093176 shutdown_connections 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- 172.21.15.99:0/2815093176 wait complete. 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.427+0000 7fb9bc475740 1 -- start start 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 Processor -- start 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901c1e0 0x564f99088b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a9cb0 con 0x564f99021d40 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a9650 con 0x564f9901cbd0 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a6b50 con 0x564f9901c1e0 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9bc475740 10 client.0 _ll_register_callbacks cb 0x7fb9ac009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9b3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901c1e0 0x564f99088b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9b2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37562/0 (socket says 172.21.15.99:37562) 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9b2000640 1 -- 172.21.15.99:0/1797589340 learned_addr learned my addr 172.21.15.99:0/1797589340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.428+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 -- 172.21.15.99:0/1797589340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901c1e0 msgr2=0x564f99088b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901c1e0 0x564f99088b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 -- 172.21.15.99:0/1797589340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 msgr2=0x564f990900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 -- 172.21.15.99:0/1797589340 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f990906c0 con 0x564f9901cbd0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9b2000640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0012050 tx=0x7fb9a0012080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17cb8d3c0a4fe98 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb99cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0024030 con 0x564f9901cbd0 2024-03-23T23:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a0005500 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.429+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9a0020770 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a0020970 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 10 client.25418 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f99091a00 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9a0005500 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9a0010430 con 0x564f9901cbd0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 10 client.25418 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 10 client.25418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 20 client.25418 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f990922d0 0x564f990946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 4 client.25418 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.430+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f99012a80 con 0x564f990922d0 2024-03-23T23:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.431+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f990922d0 0x564f990946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.431+0000 7fb9b3400640 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f990922d0 0x564f990946f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe8912dfb8859ee server_cookie=90cb60ee6d667082 in_seq=0 out_seq=0 2024-03-23T23:44:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.431+0000 7fb99cc00640 10 client.25418 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb99cc00640 1 -- 172.21.15.99:0/1797589340 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f99012a80 con 0x564f990922d0 2024-03-23T23:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb99cc00640 10 client.25418 handle_command_reply: tid=0 2024-03-23T23:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 client.25418 shutdown 2024-03-23T23:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 msgr2=0x564f99089090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0012050 tx=0x7fb9a0012080 comp rx=0 tx=0).stop 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 shutdown_connections 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f990922d0 0x564f990946f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f99021d40 0x564f990900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f9901cbd0 0x564f99089090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 --2- 172.21.15.99:0/1797589340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f9901c1e0 0x564f99088b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb99cc00640 0 client.25418 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 >> 172.21.15.99:0/1797589340 conn(0x564f99017750 msgr2=0x564f9901a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 shutdown_connections 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 1 -- 172.21.15.99:0/1797589340 wait complete. 2024-03-23T23:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.442+0000 7fb9bc475740 20 client.25418 trim_cache size 0 max 16384 2024-03-23T23:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 msgr2=0x7fb9980a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 0x7fb9980a4de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c7b0 tx=0x7fb9a40dd040 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 msgr2=0x7fb9b4197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009230 tx=0x7fb9a402c5d0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 shutdown_connections 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9980a2920 0x7fb9980a4de0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b4103290 0x7fb9b419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9b41028d0 0x7fb9b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 --2- 172.21.15.99:0/1767380541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b41016e0 0x7fb9b4197940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 >> 172.21.15.99:0/1767380541 conn(0x7fb9b40fcc30 msgr2=0x7fb9b4105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 shutdown_connections 2024-03-23T23:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:04.443+0000 7fb9b9800640 1 -- 172.21.15.99:0/1767380541 wait complete. 2024-03-23T23:44:06.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:06 smithi099 ceph-mon[31715]: pgmap v406: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:06.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:06 smithi149 ceph-mon[36083]: pgmap v406: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:06.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:06 smithi174 ceph-mon[37002]: pgmap v406: 97 pgs: 97 active+clean; 657 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:08.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:08 smithi099 ceph-mon[31715]: pgmap v407: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:08.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:08 smithi149 ceph-mon[36083]: pgmap v407: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:08.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:08 smithi174 ceph-mon[37002]: pgmap v407: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:10.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:10 smithi099 ceph-mon[31715]: pgmap v408: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T23:44:10.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:10 smithi149 ceph-mon[36083]: pgmap v408: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T23:44:10.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:10 smithi174 ceph-mon[37002]: pgmap v408: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-03-23T23:44:11.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:11 smithi149 ceph-mon[36083]: pgmap v409: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T23:44:11.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:11 smithi174 ceph-mon[37002]: pgmap v409: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T23:44:11.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:11 smithi099 ceph-mon[31715]: pgmap v409: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-03-23T23:44:14.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:13 smithi099 ceph-mon[31715]: pgmap v410: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:13 smithi149 ceph-mon[36083]: pgmap v410: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:14.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:13 smithi174 ceph-mon[37002]: pgmap v410: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:16.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:15 smithi099 ceph-mon[31715]: pgmap v411: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:15 smithi149 ceph-mon[36083]: pgmap v411: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:16.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:15 smithi174 ceph-mon[37002]: pgmap v411: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 -- 172.21.15.99:0/4141029807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 msgr2=0x7f4ffc10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 --2- 172.21.15.99:0/4141029807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc10b860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009be0 tx=0x7f4ff402bef0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 -- 172.21.15.99:0/4141029807 shutdown_connections 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 --2- 172.21.15.99:0/4141029807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc10b860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 --2- 172.21.15.99:0/4141029807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc074100 0x7f4ffc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 --2- 172.21.15.99:0/4141029807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc0756e0 0x7f4ffc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.615+0000 7f5002800640 1 -- 172.21.15.99:0/4141029807 >> 172.21.15.99:0/4141029807 conn(0x7f4ffc0fca80 msgr2=0x7f4ffc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.616+0000 7f5002800640 1 -- 172.21.15.99:0/4141029807 shutdown_connections 2024-03-23T23:44:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.616+0000 7f5002800640 1 -- 172.21.15.99:0/4141029807 wait complete. 2024-03-23T23:44:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.617+0000 7f5002800640 1 Processor -- start 2024-03-23T23:44:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.617+0000 7f5002800640 1 -- start start 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc0756e0 0x7f4ffc199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc1a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc10f3a0 con 0x7f4ffc0756e0 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc10f220 con 0x7f4ffc1093d0 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5002800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc10f520 con 0x7f4ffc074100 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5001400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5001400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42026/0 (socket says 172.21.15.99:42026) 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5001400640 1 -- 172.21.15.99:0/2584263419 learned_addr learned my addr 172.21.15.99:0/2584263419 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5000a00640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc0756e0 0x7f4ffc199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.618+0000 7f5001e00640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc1a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 -- 172.21.15.99:0/2584263419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 msgr2=0x7f4ffc1a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc1a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 -- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc0756e0 msgr2=0x7f4ffc199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc0756e0 0x7f4ffc199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 -- 172.21.15.99:0/2584263419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc1a15d0 con 0x7f4ffc074100 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5001400640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000e9c0 tx=0x7f4ff000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f4ffa000640 1 -- 172.21.15.99:0/2584263419 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000cde0 con 0x7f4ffc074100 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f4ffa000640 1 -- 172.21.15.99:0/2584263419 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff00035f0 con 0x7f4ffc074100 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f4ffa000640 1 -- 172.21.15.99:0/2584263419 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0010690 con 0x7f4ffc074100 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.619+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc1a18c0 con 0x7f4ffc074100 2024-03-23T23:44:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.620+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc1a1e60 con 0x7f4ffc074100 2024-03-23T23:44:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 Processor -- start 2024-03-23T23:44:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 -- start start 2024-03-23T23:44:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f4ffa000640 1 -- 172.21.15.99:0/2584263419 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4ff00037a0 con 0x7f4ffc074100 2024-03-23T23:44:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfd68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfd93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796df943b0 con 0x55796dfd1de0 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796df36830 con 0x55796dfd6f50 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5006bcc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796dedf5d0 con 0x55796dfd13f0 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5001e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5001e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42032/0 (socket says 172.21.15.99:42032) 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.623+0000 7f5001e00640 1 -- 172.21.15.99:0/471701244 learned_addr learned my addr 172.21.15.99:0/471701244 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5000a00640 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfd93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001400640 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfd68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 -- 172.21.15.99:0/471701244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 msgr2=0x55796dfd93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfd93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 -- 172.21.15.99:0/471701244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 msgr2=0x55796dfd68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfd68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 -- 172.21.15.99:0/471701244 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796def72c0 con 0x55796dfd13f0 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f5001e00640 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ff402c500 tx=0x7f4ff4009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9864719f73326f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f4fee600640 1 -- 172.21.15.99:0/471701244 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4043030 con 0x55796dfd13f0 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f4fee600640 1 -- 172.21.15.99:0/471701244 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff4033bc0 con 0x55796dfd13f0 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.624+0000 7f4fee600640 1 -- 172.21.15.99:0/471701244 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff402c980 con 0x55796dfd13f0 2024-03-23T23:44:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f4ffa000640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 0x7f4fe40a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f4ffa000640 1 -- 172.21.15.99:0/2584263419 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ff00ced20 con 0x7f4ffc074100 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5001400640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 0x7f4fe40a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 -- 172.21.15.99:0/471701244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 msgr2=0x55796dfd1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ff402c500 tx=0x7f4ff4009330 comp rx=0 tx=0).stop 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 -- 172.21.15.99:0/471701244 shutdown_connections 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfd93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfd68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 --2- 172.21.15.99:0/471701244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfd1810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 -- 172.21.15.99:0/471701244 >> 172.21.15.99:0/471701244 conn(0x55796dfcc960 msgr2=0x55796dfccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.625+0000 7f5006bcc740 1 -- 172.21.15.99:0/471701244 shutdown_connections 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.626+0000 7f5006bcc740 1 -- 172.21.15.99:0/471701244 wait complete. 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.626+0000 7f5001400640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 0x7f4fe40a4d30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0097aa0 tx=0x7f4ff0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.626+0000 7f5006bcc740 1 -- start start 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.626+0000 7f5006bcc740 1 Processor -- start 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd6f50 0x55796dfe2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796def2a20 con 0x55796dfd13f0 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796df328d0 con 0x55796dfd1de0 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796dfd9ea0 con 0x55796dfd6f50 2024-03-23T23:44:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5006bcc740 10 client.0 _ll_register_callbacks cb 0x55796df2f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5001e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5001e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47136/0 (socket says 172.21.15.99:47136) 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5001e00640 1 -- 172.21.15.99:0/302278608 learned_addr learned my addr 172.21.15.99:0/302278608 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56208/0 (socket says 172.21.15.99:56208) 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5001e00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd6f50 0x55796dfe2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 -- 172.21.15.99:0/302278608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd6f50 msgr2=0x55796dfe2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd6f50 0x55796dfe2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 -- 172.21.15.99:0/302278608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 msgr2=0x55796dfdb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5000a00640 1 -- 172.21.15.99:0/302278608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55796dc4b210 con 0x55796dfd1de0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.627+0000 7f5001e00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f5000a00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc19aec0 tx=0x7f4fe8012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b957852fdc76db2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f4fe2a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8005220 con 0x55796dfd1de0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff4033bc0 con 0x55796dfd1de0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4fe80053c0 con 0x55796dfd1de0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe80055c0 con 0x55796dfd1de0 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f5006bcc740 10 client.25430 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55796dd17620 con 0x55796dfd1de0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.628+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff4033bc0 con 0x55796dfd1de0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4fe8010c00 con 0x55796dfd1de0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 10 client.25430 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 20 client.25430 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796dfe73d0 0x55796dfe9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 4 client.25430 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55796df43d70 con 0x55796dfe73d0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5001e00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796dfe73d0 0x55796dfe9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f5001e00640 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796dfe73d0 0x55796dfe9810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9460095e8890748e server_cookie=9bee141ee7201973 in_seq=0 out_seq=0 2024-03-23T23:44:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.629+0000 7f4fe2a00640 10 client.25430 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f4fe2a00640 1 -- 172.21.15.99:0/302278608 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55796df43d70 con 0x55796dfe73d0 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f4fe2a00640 10 client.25430 handle_command_reply: tid=0 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 client.25430 shutdown 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 msgr2=0x55796dfdbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc19aec0 tx=0x7f4fe8012820 comp rx=0 tx=0).stop 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 shutdown_connections 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796dfe73d0 0x55796dfe9810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd6f50 0x55796dfe2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd1de0 0x55796dfdbab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f4fe2a00640 0 client.25430 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 --2- 172.21.15.99:0/302278608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.630+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 >> 172.21.15.99:0/302278608 conn(0x55796dfcc960 msgr2=0x55796dfcbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.631+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 shutdown_connections 2024-03-23T23:44:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.631+0000 7f5006bcc740 1 -- 172.21.15.99:0/302278608 wait complete. 2024-03-23T23:44:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.631+0000 7f5006bcc740 20 client.25430 trim_cache size 0 max 16384 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 Processor -- start 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 -- start start 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfdbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1aa990 con 0x55796dfd13f0 2024-03-23T23:44:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1ac310 con 0x55796dfd6f50 2024-03-23T23:44:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5006bcc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1aaff0 con 0x55796dfd1de0 2024-03-23T23:44:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5000a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5001e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5000a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42056/0 (socket says 172.21.15.99:42056) 2024-03-23T23:44:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5000a00640 1 -- 172.21.15.99:0/2507481291 learned_addr learned my addr 172.21.15.99:0/2507481291 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.646+0000 7f5001e00640 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfdbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 -- 172.21.15.99:0/2507481291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 msgr2=0x55796dfdbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfdbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 -- 172.21.15.99:0/2507481291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 msgr2=0x55796dfdb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 -- 172.21.15.99:0/2507481291 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796dfd9fe0 con 0x55796dfd1de0 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f5000a00640 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8013fd0 tx=0x7f4fe8005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9b6aa3e2811a27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f4fedc00640 1 -- 172.21.15.99:0/2507481291 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8000e60 con 0x55796dfd1de0 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f4fedc00640 1 -- 172.21.15.99:0/2507481291 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8005220 con 0x55796dfd1de0 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.647+0000 7f4fedc00640 1 -- 172.21.15.99:0/2507481291 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8004660 con 0x55796dfd1de0 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 -- 172.21.15.99:0/2507481291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 msgr2=0x55796dfe2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8013fd0 tx=0x7f4fe8005d40 comp rx=0 tx=0).stop 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 -- 172.21.15.99:0/2507481291 shutdown_connections 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfdbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd1de0 0x55796dfe2ab0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2507481291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd13f0 0x55796dfdb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 -- 172.21.15.99:0/2507481291 >> 172.21.15.99:0/2507481291 conn(0x55796dfcc960 msgr2=0x55796dfccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 -- 172.21.15.99:0/2507481291 shutdown_connections 2024-03-23T23:44:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.648+0000 7f5006bcc740 1 -- 172.21.15.99:0/2507481291 wait complete. 2024-03-23T23:44:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.649+0000 7f5006bcc740 1 -- start start 2024-03-23T23:44:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.649+0000 7f5006bcc740 1 Processor -- start 2024-03-23T23:44:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.649+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfe0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfe1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1abcb0 con 0x55796dfd1de0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1ab650 con 0x55796dfd6f50 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1a8b50 con 0x55796dfd13f0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5006bcc740 10 client.0 _ll_register_callbacks cb 0x7f4fe800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:42060/0 (socket says 172.21.15.99:42060) 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 -- 172.21.15.99:0/2589002764 learned_addr learned my addr 172.21.15.99:0/2589002764 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfe1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5000a00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfe0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 -- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 msgr2=0x55796dfe1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfe1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 -- 172.21.15.99:0/2589002764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 msgr2=0x55796dfe0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfe0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 -- 172.21.15.99:0/2589002764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55796dfe2210 con 0x55796dfd13f0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f5001e00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4003cc0 tx=0x7f4ff4034f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d13709fda42daea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.650+0000 7f4fe2a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4043040 con 0x55796dfd13f0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ff402c890 con 0x55796dfd13f0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fe8005220 con 0x55796dfd13f0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff4004b40 con 0x55796dfd13f0 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 10 client.25462 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55796e0431e0 con 0x55796dfd13f0 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8005220 con 0x55796dfd13f0 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4ff4055a80 con 0x55796dfd13f0 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 10 client.25462 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.651+0000 7f5006bcc740 20 client.25462 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796e0436d0 0x55796e045ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f5006bcc740 4 client.25462 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55796df3e0c0 con 0x55796e0436d0 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f5000a00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796e0436d0 0x55796e045ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f4fe2a00640 10 client.25462 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.652+0000 7f5000a00640 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796e0436d0 0x55796e045ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabc6f11a751c210 server_cookie=6cd20f46f28b7764 in_seq=0 out_seq=0 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f4fe2a00640 1 -- 172.21.15.99:0/2589002764 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55796df3e0c0 con 0x55796e0436d0 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f4fe2a00640 10 client.25462 handle_command_reply: tid=0 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 client.25462 shutdown 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 msgr2=0x55796dfe06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4003cc0 tx=0x7f4ff4034f90 comp rx=0 tx=0).stop 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 shutdown_connections 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55796e0436d0 0x55796e045ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55796dfd6f50 0x55796dfe1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55796dfd1de0 0x55796dfe0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f4fe2a00640 0 client.25462 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 --2- 172.21.15.99:0/2589002764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55796dfd13f0 0x55796dfe06d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.658+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 >> 172.21.15.99:0/2589002764 conn(0x55796dfcc960 msgr2=0x55796dfd3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 shutdown_connections 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5006bcc740 1 -- 172.21.15.99:0/2589002764 wait complete. 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5006bcc740 20 client.25462 trim_cache size 0 max 16384 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:44:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 msgr2=0x7f4fe40a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 0x7f4fe40a4d30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0097aa0 tx=0x7f4ff0000bc0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 msgr2=0x7f4ffc1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.659+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000e9c0 tx=0x7f4ff000ee80 comp rx=0 tx=0).stop 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 shutdown_connections 2024-03-23T23:44:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4fe40a2870 0x7f4fe40a4d30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ffc1093d0 0x7f4ffc1a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ffc0756e0 0x7f4ffc199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 --2- 172.21.15.99:0/2584263419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ffc074100 0x7f4ffc1999b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 >> 172.21.15.99:0/2584263419 conn(0x7f4ffc0fca80 msgr2=0x7f4ffc1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 shutdown_connections 2024-03-23T23:44:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:17.660+0000 7f5002800640 1 -- 172.21.15.99:0/2584263419 wait complete. 2024-03-23T23:44:18.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:17 smithi099 ceph-mon[31715]: pgmap v412: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:17 smithi149 ceph-mon[36083]: pgmap v412: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:18.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:17 smithi174 ceph-mon[37002]: pgmap v412: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-23T23:44:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:19 smithi149 ceph-mon[36083]: pgmap v413: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-23T23:44:20.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:19 smithi174 ceph-mon[37002]: pgmap v413: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-23T23:44:20.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:19 smithi099 ceph-mon[31715]: pgmap v413: 97 pgs: 97 active+clean; 661 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-23T23:44:21.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:44:21.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 -- 172.21.15.99:0/5480622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc470103150 msgr2=0x7fc4701099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 --2- 172.21.15.99:0/5480622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc470103150 0x7fc4701099e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc468009be0 tx=0x7fc46802bef0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 -- 172.21.15.99:0/5480622 shutdown_connections 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 --2- 172.21.15.99:0/5480622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc470103150 0x7fc4701099e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 --2- 172.21.15.99:0/5480622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470102790 0x7fc470102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 --2- 172.21.15.99:0/5480622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4701015a0 0x7fc4701019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.969+0000 7fc475200640 1 -- 172.21.15.99:0/5480622 >> 172.21.15.99:0/5480622 conn(0x7fc4700fcb10 msgr2=0x7fc4700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:21.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.970+0000 7fc475200640 1 -- 172.21.15.99:0/5480622 shutdown_connections 2024-03-23T23:44:21.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.970+0000 7fc475200640 1 -- 172.21.15.99:0/5480622 wait complete. 2024-03-23T23:44:21.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.971+0000 7fc475200640 1 Processor -- start 2024-03-23T23:44:21.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.971+0000 7fc475200640 1 -- start start 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4701015a0 0x7fc470199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 0x7fc4701a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010d250 con 0x7fc4701015a0 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010d0d0 con 0x7fc470103150 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc475200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010d3d0 con 0x7fc470102790 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc46f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4701015a0 0x7fc470199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc46fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 0x7fc4701a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48492/0 (socket says 172.21.15.99:48492) 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.972+0000 7fc467e00640 1 -- 172.21.15.99:0/1667963196 learned_addr learned my addr 172.21.15.99:0/1667963196 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 -- 172.21.15.99:0/1667963196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 msgr2=0x7fc4701a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 0x7fc4701a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 -- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4701015a0 msgr2=0x7fc470199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:21.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4701015a0 0x7fc470199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 -- 172.21.15.99:0/1667963196 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4701a1640 con 0x7fc470102790 2024-03-23T23:44:21.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc46fe00640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 0x7fc4701a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:21.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc467e00640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9f0 tx=0x7fc45800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:21.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc46cc00640 1 -- 172.21.15.99:0/1667963196 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ccc0 con 0x7fc470102790 2024-03-23T23:44:21.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc46cc00640 1 -- 172.21.15.99:0/1667963196 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458010910 con 0x7fc470102790 2024-03-23T23:44:21.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc46cc00640 1 -- 172.21.15.99:0/1667963196 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580188a0 con 0x7fc470102790 2024-03-23T23:44:21.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4701a1930 con 0x7fc470102790 2024-03-23T23:44:21.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.973+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4701a1e30 con 0x7fc470102790 2024-03-23T23:44:21.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.976+0000 7fc46cc00640 1 -- 172.21.15.99:0/1667963196 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc458010ac0 con 0x7fc470102790 2024-03-23T23:44:21.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc46cc00640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 0x7fc4540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc46cc00640 1 -- 172.21.15.99:0/1667963196 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc4580cee80 con 0x7fc470102790 2024-03-23T23:44:21.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc4793fc740 1 Processor -- start 2024-03-23T23:44:21.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc46f400640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 0x7fc4540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc4793fc740 1 -- start start 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.978+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b46770c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46778850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc4793fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b466cd250 con 0x556b467763c0 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc4793fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b46675a30 con 0x556b46770860 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc4793fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b466c92f0 con 0x556b46771250 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc46f400640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 0x7fc4540a4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc4701025f0 tx=0x7fc460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc46f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b46770c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48506/0 (socket says 172.21.15.99:48506) 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 -- 172.21.15.99:0/3941756583 learned_addr learned my addr 172.21.15.99:0/3941756583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc46fe00640 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46778850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 -- 172.21.15.99:0/3941756583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 msgr2=0x556b46770c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b46770c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 -- 172.21.15.99:0/3941756583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 msgr2=0x556b46778850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46778850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc467e00640 1 -- 172.21.15.99:0/3941756583 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b46660690 con 0x556b46771250 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.979+0000 7fc46f400640 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b46770c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.980+0000 7fc467e00640 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc45809b730 tx=0x7fc45800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c355d5fbe352e60d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.980+0000 7fc464c00640 1 -- 172.21.15.99:0/3941756583 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580a3f40 con 0x556b46771250 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.980+0000 7fc464c00640 1 -- 172.21.15.99:0/3941756583 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458099aa0 con 0x556b46771250 2024-03-23T23:44:21.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.980+0000 7fc464c00640 1 -- 172.21.15.99:0/3941756583 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580974f0 con 0x556b46771250 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 -- 172.21.15.99:0/3941756583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 msgr2=0x556b46775d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc45809b730 tx=0x7fc45800ce60 comp rx=0 tx=0).stop 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 -- 172.21.15.99:0/3941756583 shutdown_connections 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46778850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b46775d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 --2- 172.21.15.99:0/3941756583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b46770c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 -- 172.21.15.99:0/3941756583 >> 172.21.15.99:0/3941756583 conn(0x556b46730200 msgr2=0x556b467639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 -- 172.21.15.99:0/3941756583 shutdown_connections 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.981+0000 7fc4793fc740 1 -- 172.21.15.99:0/3941756583 wait complete. 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 -- start start 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 Processor -- start 2024-03-23T23:44:21.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46770860 0x556b4677a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46781c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b467792b0 con 0x556b467763c0 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b46688e80 con 0x556b46771250 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b46779430 con 0x556b46770860 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.982+0000 7fc4793fc740 10 client.0 _ll_register_callbacks cb 0x556b4669b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55932/0 (socket says 172.21.15.99:55932) 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 -- 172.21.15.99:0/2459025460 learned_addr learned my addr 172.21.15.99:0/2459025460 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc46f400640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46770860 0x556b4677a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc46fe00640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46781c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 -- 172.21.15.99:0/2459025460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46770860 msgr2=0x556b4677a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46770860 0x556b4677a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 -- 172.21.15.99:0/2459025460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 msgr2=0x556b46781c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46781c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 -- 172.21.15.99:0/2459025460 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b464ab7d0 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc467e00640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc45809e040 tx=0x7fc4580862f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7013da168e584936 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc45d600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.983+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580b8070 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc4580afd90 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc458099aa0 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc458097ec0 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 10 client.25448 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b463e13f0 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458099aa0 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc45809cb60 con 0x556b46771250 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 10 client.25448 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 10 client.25448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:21.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 20 client.25448 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:21.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b4676bd00 0x556b4676e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:21.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 4 client.25448 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:21.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.984+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b466da790 con 0x556b4676bd00 2024-03-23T23:44:21.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.985+0000 7fc46f400640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b4676bd00 0x556b4676e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.985+0000 7fc45d600640 10 client.25448 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.985+0000 7fc46f400640 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b4676bd00 0x556b4676e120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8888f37b8c30ebcb server_cookie=98a4aa5a62c7f5c7 in_seq=0 out_seq=0 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc45d600640 1 -- 172.21.15.99:0/2459025460 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556b466da790 con 0x556b4676bd00 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc45d600640 10 client.25448 handle_command_reply: tid=0 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc4793fc740 1 client.25448 shutdown 2024-03-23T23:44:21.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 msgr2=0x556b4677ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc45809e040 tx=0x7fc4580862f0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.986+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 shutdown_connections 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b4676bd00 0x556b4676e120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46781c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b4677ac30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2459025460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46770860 0x556b4677a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 >> 172.21.15.99:0/2459025460 conn(0x556b46730200 msgr2=0x556b46762bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc45d600640 0 client.25448 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 shutdown_connections 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 1 -- 172.21.15.99:0/2459025460 wait complete. 2024-03-23T23:44:21.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:21.987+0000 7fc4793fc740 20 client.25448 trim_cache size 0 max 16384 2024-03-23T23:44:22.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 Processor -- start 2024-03-23T23:44:22.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 -- start start 2024-03-23T23:44:22.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b46770860 0x556b467723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b46781c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 0x556b46766c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a66c0 con 0x556b46770860 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a8040 con 0x556b46771250 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc4793fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a6d20 con 0x556b467763c0 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc46fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 0x556b46766c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc467e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b46781c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc46f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b46770860 0x556b467723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.004+0000 7fc46fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 0x556b46766c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48522/0 (socket says 172.21.15.99:48522) 2024-03-23T23:44:22.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc46fe00640 1 -- 172.21.15.99:0/2075904811 learned_addr learned my addr 172.21.15.99:0/2075904811 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 -- 172.21.15.99:0/2075904811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 msgr2=0x556b46766c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 0x556b46766c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 -- 172.21.15.99:0/2075904811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b46770860 msgr2=0x556b467723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b46770860 0x556b467723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 -- 172.21.15.99:0/2075904811 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b463a10f0 con 0x556b46771250 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc467e00640 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b46781c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc458086360 tx=0x7fc45800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d0289420f33fee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc45fe00640 1 -- 172.21.15.99:0/2075904811 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580ae070 con 0x556b46771250 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc45fe00640 1 -- 172.21.15.99:0/2075904811 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45809cad0 con 0x556b46771250 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.005+0000 7fc45fe00640 1 -- 172.21.15.99:0/2075904811 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580b8070 con 0x556b46771250 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 -- 172.21.15.99:0/2075904811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 msgr2=0x556b46781c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b46781c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc458086360 tx=0x7fc45800ce60 comp rx=0 tx=0).stop 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 -- 172.21.15.99:0/2075904811 shutdown_connections 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b467763c0 0x556b46766c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46771250 0x556b46781c30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 --2- 172.21.15.99:0/2075904811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b46770860 0x556b467723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 -- 172.21.15.99:0/2075904811 >> 172.21.15.99:0/2075904811 conn(0x556b46730200 msgr2=0x556b46771cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 -- 172.21.15.99:0/2075904811 shutdown_connections 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.006+0000 7fc4793fc740 1 -- 172.21.15.99:0/2075904811 wait complete. 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 -- start start 2024-03-23T23:44:22.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 Processor -- start 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46780bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a79e0 con 0x556b467763c0 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a7380 con 0x556b46770860 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701a4880 con 0x556b46771250 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.007+0000 7fc4793fc740 10 client.0 _ll_register_callbacks cb 0x7fc460009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc46fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46780bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc46f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48530/0 (socket says 172.21.15.99:48530) 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 -- 172.21.15.99:0/12501223 learned_addr learned my addr 172.21.15.99:0/12501223 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc46f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55948/0 (socket says 172.21.15.99:55948) 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 -- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 msgr2=0x556b4677d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 -- 172.21.15.99:0/12501223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 msgr2=0x556b46780bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46780bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 -- 172.21.15.99:0/12501223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b467811c0 con 0x556b46771250 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc46f400640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc467e00640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc458097c00 tx=0x7fc45800cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9785dd25ccdec5f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc45d600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.008+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580ae030 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc458099aa0 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc45809cad0 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4580b8070 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 10 client.25480 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556b467ea100 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45809cad0 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fc4580a0a20 con 0x556b46771250 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 10 client.25480 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 20 client.25480 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b467ea630 0x556b467eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 4 client.25480 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.009+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556b4677a4e0 con 0x556b467ea630 2024-03-23T23:44:22.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc46f400640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b467ea630 0x556b467eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc45d600640 10 client.25480 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc46f400640 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b467ea630 0x556b467eca50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d8753ad2912a52 server_cookie=82941afc88a153f9 in_seq=0 out_seq=0 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc45d600640 1 -- 172.21.15.99:0/12501223 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556b4677a4e0 con 0x556b467ea630 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc45d600640 10 client.25480 handle_command_reply: tid=0 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.010+0000 7fc4793fc740 1 client.25480 shutdown 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 msgr2=0x556b4677dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc458097c00 tx=0x7fc45800cef0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 shutdown_connections 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x556b467ea630 0x556b467eca50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b467763c0 0x556b46780bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc45d600640 0 client.25480 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b46771250 0x556b4677dca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 --2- 172.21.15.99:0/12501223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b46770860 0x556b4677d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 >> 172.21.15.99:0/12501223 conn(0x556b46730200 msgr2=0x556b46762bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 shutdown_connections 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 1 -- 172.21.15.99:0/12501223 wait complete. 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.011+0000 7fc4793fc740 20 client.25480 trim_cache size 0 max 16384 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-23T23:44:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 msgr2=0x7fc4540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 0x7fc4540a4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc4701025f0 tx=0x7fc460009260 comp rx=0 tx=0).stop 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 msgr2=0x7fc470199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc45800e9f0 tx=0x7fc45800edf0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 shutdown_connections 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fc4540a2880 0x7fc4540a4d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc470103150 0x7fc4701a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102790 0x7fc470199f60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 --2- 172.21.15.99:0/1667963196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4701015a0 0x7fc470199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 >> 172.21.15.99:0/1667963196 conn(0x7fc4700fcb10 msgr2=0x7fc4700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 shutdown_connections 2024-03-23T23:44:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.012+0000 7fc475200640 1 -- 172.21.15.99:0/1667963196 wait complete. 2024-03-23T23:44:22.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T23:44:22.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:44:22.098 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:21 smithi099 ceph-mon[31715]: pgmap v414: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-23T23:44:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:21 smithi149 ceph-mon[36083]: pgmap v414: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-23T23:44:22.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:21 smithi174 ceph-mon[37002]: pgmap v414: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-23T23:44:22.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 -- 172.21.15.99:0/367287531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3844103700 msgr2=0x7f3844103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 --2- 172.21.15.99:0/367287531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3844103700 0x7f3844103b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f383402bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 -- 172.21.15.99:0/367287531 shutdown_connections 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 --2- 172.21.15.99:0/367287531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38441052b0 0x7f384410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 --2- 172.21.15.99:0/367287531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 0x7f3844104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 --2- 172.21.15.99:0/367287531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3844103700 0x7f3844103b20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 -- 172.21.15.99:0/367287531 >> 172.21.15.99:0/367287531 conn(0x7f38440fec70 msgr2=0x7f38441010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 -- 172.21.15.99:0/367287531 shutdown_connections 2024-03-23T23:44:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.223+0000 7f3848e00640 1 -- 172.21.15.99:0/367287531 wait complete. 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.224+0000 7f3848e00640 1 Processor -- start 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.224+0000 7f3848e00640 1 -- start start 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.224+0000 7f3848e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3848e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 0x7f3844199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3848e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 0x7f38441a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3848e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384410f3a0 con 0x7f38441048f0 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3848e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384410f220 con 0x7f38441052b0 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3848e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384410f520 con 0x7f3844103700 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48548/0 (socket says 172.21.15.99:48548) 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 -- 172.21.15.99:0/2212344183 learned_addr learned my addr 172.21.15.99:0/2212344183 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f383be00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 0x7f3844199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 0x7f38441a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 -- 172.21.15.99:0/2212344183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 msgr2=0x7f38441a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 0x7f38441a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 -- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 msgr2=0x7f3844199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 0x7f3844199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843400640 1 -- 172.21.15.99:0/2212344183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38441a1660 con 0x7f3844103700 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.225+0000 7f3843e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 0x7f38441a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3843400640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f383402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834043020 con 0x7f3844103700 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38340054e0 con 0x7f3844103700 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38441a18f0 con 0x7f3844103700 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834003c40 con 0x7f3844103700 2024-03-23T23:44:22.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.226+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38441a1e90 con 0x7f3844103700 2024-03-23T23:44:22.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.227+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384410aa70 con 0x7f3844103700 2024-03-23T23:44:22.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.229+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3834046020 con 0x7f3844103700 2024-03-23T23:44:22.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.231+0000 7f3840c00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 0x7f38280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.232+0000 7f383be00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 0x7f38280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.232+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38340f0b20 con 0x7f3844103700 2024-03-23T23:44:22.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.232+0000 7f383be00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 0x7f38280a4d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f384419af50 tx=0x7f382c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.234+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f383404d020 con 0x7f3844103700 2024-03-23T23:44:22.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.488+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384402d190 con 0x7f3844103700 2024-03-23T23:44:22.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.489+0000 7f3840c00640 1 -- 172.21.15.99:0/2212344183 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f38340bdb40 con 0x7f3844103700 2024-03-23T23:44:22.490 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:44:22.490 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:44:22.490 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 msgr2=0x7f38280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 0x7f38280a4d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f384419af50 tx=0x7f382c005f90 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 msgr2=0x7f3844199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3834000c00 tx=0x7f383402c650 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 shutdown_connections 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38280a2880 0x7f38280a4d40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38441052b0 0x7f38441a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38441048f0 0x7f3844199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 --2- 172.21.15.99:0/2212344183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3844103700 0x7f3844199a40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 >> 172.21.15.99:0/2212344183 conn(0x7f38440fec70 msgr2=0x7f3844107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 shutdown_connections 2024-03-23T23:44:22.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.492+0000 7f3848e00640 1 -- 172.21.15.99:0/2212344183 wait complete. 2024-03-23T23:44:22.511 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:22.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T23:44:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.699+0000 7f959b000640 1 -- 172.21.15.99:0/2736580270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95940772a0 msgr2=0x7f9594077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.699+0000 7f959b000640 1 --2- 172.21.15.99:0/2736580270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95940772a0 0x7f9594077700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f957c009860 tx=0x7f957c02bb70 comp rx=0 tx=0).stop 2024-03-23T23:44:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 -- 172.21.15.99:0/2736580270 shutdown_connections 2024-03-23T23:44:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 --2- 172.21.15.99:0/2736580270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9594077c40 0x7f95940784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 --2- 172.21.15.99:0/2736580270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95940772a0 0x7f9594077700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 --2- 172.21.15.99:0/2736580270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594076060 0x7f9594076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 -- 172.21.15.99:0/2736580270 >> 172.21.15.99:0/2736580270 conn(0x7f95940fee50 msgr2=0x7f9594101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 -- 172.21.15.99:0/2736580270 shutdown_connections 2024-03-23T23:44:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.700+0000 7f959b000640 1 -- 172.21.15.99:0/2736580270 wait complete. 2024-03-23T23:44:22.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 Processor -- start 2024-03-23T23:44:22.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 -- start start 2024-03-23T23:44:22.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940772a0 0x7f959419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 0x7f95941a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594113970 con 0x7f95940772a0 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941137f0 con 0x7f9594076060 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f959b000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594113af0 con 0x7f9594077c40 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55986/0 (socket says 172.21.15.99:55986) 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f9593e00640 1 -- 172.21.15.99:0/2918655891 learned_addr learned my addr 172.21.15.99:0/2918655891 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f9599000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 0x7f95941a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.701+0000 7f9593400640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940772a0 0x7f959419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 -- 172.21.15.99:0/2918655891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 msgr2=0x7f95941a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 0x7f95941a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 -- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940772a0 msgr2=0x7f959419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940772a0 0x7f959419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 -- 172.21.15.99:0/2918655891 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95941a5bc0 con 0x7f9594076060 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9599000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 0x7f95941a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:22.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9593e00640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f958400c420 tx=0x7f9584009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.702+0000 7f9590c00640 1 -- 172.21.15.99:0/2918655891 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584003970 con 0x7f9594076060 2024-03-23T23:44:22.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.703+0000 7f9590c00640 1 -- 172.21.15.99:0/2918655891 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9584004020 con 0x7f9594076060 2024-03-23T23:44:22.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.703+0000 7f9590c00640 1 -- 172.21.15.99:0/2918655891 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400bad0 con 0x7f9594076060 2024-03-23T23:44:22.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.703+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95941a5eb0 con 0x7f9594076060 2024-03-23T23:44:22.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.703+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95941a62c0 con 0x7f9594076060 2024-03-23T23:44:22.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.706+0000 7f959de2f740 1 Processor -- start 2024-03-23T23:44:22.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.706+0000 7f959de2f740 1 -- start start 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8663e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86643a0 0x55c7b866ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b85c72c0 con 0x55c7b86639e0 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b856fc90 con 0x55c7b86643a0 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f959de2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b85c3360 con 0x55c7b8662810 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48576/0 (socket says 172.21.15.99:48576) 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 -- 172.21.15.99:0/2644054201 learned_addr learned my addr 172.21.15.99:0/2644054201 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9593400640 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86643a0 0x55c7b866ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 -- 172.21.15.99:0/2644054201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86643a0 msgr2=0x55c7b866ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86643a0 0x55c7b866ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 -- 172.21.15.99:0/2644054201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 msgr2=0x55c7b8663e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8663e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9590c00640 1 -- 172.21.15.99:0/2918655891 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f958401d020 con 0x7f9594076060 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.707+0000 7f9599000640 1 -- 172.21.15.99:0/2644054201 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b8594080 con 0x55c7b8662810 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.708+0000 7f9599000640 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f958c011540 tx=0x7f958c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d41a76e1da7f6d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.708+0000 7f9589600640 1 -- 172.21.15.99:0/2644054201 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00f5c0 con 0x55c7b8662810 2024-03-23T23:44:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.708+0000 7f9589600640 1 -- 172.21.15.99:0/2644054201 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f958c00fc40 con 0x55c7b8662810 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.708+0000 7f9589600640 1 -- 172.21.15.99:0/2644054201 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c019d20 con 0x55c7b8662810 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- 172.21.15.99:0/2644054201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 msgr2=0x55c7b8662c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f958c011540 tx=0x7f958c011a00 comp rx=0 tx=0).stop 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- 172.21.15.99:0/2644054201 shutdown_connections 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86643a0 0x55c7b866ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8663e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 --2- 172.21.15.99:0/2644054201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b8662c30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- 172.21.15.99:0/2644054201 >> 172.21.15.99:0/2644054201 conn(0x55c7b865dc00 msgr2=0x55c7b865e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- 172.21.15.99:0/2644054201 shutdown_connections 2024-03-23T23:44:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- 172.21.15.99:0/2644054201 wait complete. 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f959de2f740 1 -- start start 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.709+0000 7f9590c00640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 0x7f95780a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f9590c00640 1 -- 172.21.15.99:0/2918655891 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f95840ce8c0 con 0x7f9594076060 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 Processor -- start 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f9599000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 0x7f95780a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86639e0 0x55c7b866cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b8674010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b866b3b0 con 0x55c7b86643a0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b85830d0 con 0x55c7b86639e0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.710+0000 7f959de2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b866b530 con 0x55c7b8662810 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48582/0 (socket says 172.21.15.99:48582) 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 -- 172.21.15.99:0/2487857437 learned_addr learned my addr 172.21.15.99:0/2487857437 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f959de2f740 10 client.0 _ll_register_callbacks cb 0x55c7b8587e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593400640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86639e0 0x55c7b866cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9599000640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b8674010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 -- 172.21.15.99:0/2487857437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86639e0 msgr2=0x55c7b866cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86639e0 0x55c7b866cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 -- 172.21.15.99:0/2487857437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 msgr2=0x55c7b8674010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b8674010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 -- 172.21.15.99:0/2487857437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7b82db3f0 con 0x55c7b8662810 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9599000640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b8674010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9599000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 0x7f95780a4c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f958c009ea0 tx=0x7f958c009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9593e00640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9584097bb0 tx=0x7f9584004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4103a54328a057a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9582000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958409f120 con 0x55c7b8662810 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f95840041d0 con 0x55c7b8662810 2024-03-23T23:44:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.711+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f958c00fc40 con 0x55c7b8662810 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f958409b1e0 con 0x55c7b8662810 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 10 client.25498 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c7b83a57d0 con 0x55c7b8662810 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f958c00fc40 con 0x55c7b8662810 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f95840b9ce0 con 0x55c7b8662810 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 10 client.25498 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 10 client.25498 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 20 client.25498 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86784f0 0x55c7b867a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 4 client.25498 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.712+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7b85d4800 con 0x55c7b86784f0 2024-03-23T23:44:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.713+0000 7f9593400640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86784f0 0x55c7b867a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.713+0000 7f9593400640 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86784f0 0x55c7b867a920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f45e6ab0f403adf server_cookie=62020150db423f8b in_seq=0 out_seq=0 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.713+0000 7f9582000640 10 client.25498 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f9582000640 1 -- 172.21.15.99:0/2487857437 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c7b85d4800 con 0x55c7b86784f0 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f9582000640 10 client.25498 handle_command_reply: tid=0 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f959de2f740 1 client.25498 shutdown 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 msgr2=0x55c7b866ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9584097bb0 tx=0x7f9584004600 comp rx=0 tx=0).stop 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 shutdown_connections 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.714+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86784f0 0x55c7b867a920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b8674010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b86639e0 0x55c7b866cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f9582000640 0 client.25498 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 --2- 172.21.15.99:0/2487857437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b8662810 0x55c7b866ca90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 >> 172.21.15.99:0/2487857437 conn(0x55c7b865dc00 msgr2=0x55c7b865ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 shutdown_connections 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 1 -- 172.21.15.99:0/2487857437 wait complete. 2024-03-23T23:44:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.715+0000 7f959de2f740 20 client.25498 trim_cache size 0 max 16384 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 Processor -- start 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 -- start start 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b867f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.730+0000 7f959de2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941af010 con 0x55c7b86643a0 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f959de2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941b0990 con 0x55c7b8662810 2024-03-23T23:44:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f959de2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941af670 con 0x55c7b86639e0 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9599000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b867f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56016/0 (socket says 172.21.15.99:56016) 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48594/0 (socket says 172.21.15.99:48594) 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 -- 172.21.15.99:0/3853232639 learned_addr learned my addr 172.21.15.99:0/3853232639 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 -- 172.21.15.99:0/3853232639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 msgr2=0x55c7b8673fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 -- 172.21.15.99:0/3853232639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 msgr2=0x55c7b867f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b867f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593e00640 1 -- 172.21.15.99:0/3853232639 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b866af60 con 0x55c7b8662810 2024-03-23T23:44:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.731+0000 7f9593400640 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.732+0000 7f9593e00640 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f958400c3f0 tx=0x7f95840996f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca67c3bd9b5ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.732+0000 7f9588c00640 1 -- 172.21.15.99:0/3853232639 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958409a800 con 0x55c7b8662810 2024-03-23T23:44:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.732+0000 7f9588c00640 1 -- 172.21.15.99:0/3853232639 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f958409f330 con 0x55c7b8662810 2024-03-23T23:44:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.732+0000 7f9588c00640 1 -- 172.21.15.99:0/3853232639 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584004360 con 0x55c7b8662810 2024-03-23T23:44:22.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- 172.21.15.99:0/3853232639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 msgr2=0x55c7b867f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f958400c3f0 tx=0x7f95840996f0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- 172.21.15.99:0/3853232639 shutdown_connections 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86643a0 0x55c7b867f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86639e0 0x55c7b8673fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 --2- 172.21.15.99:0/3853232639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b867f180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- 172.21.15.99:0/3853232639 >> 172.21.15.99:0/3853232639 conn(0x55c7b865dc00 msgr2=0x55c7b86616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- 172.21.15.99:0/3853232639 shutdown_connections 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- 172.21.15.99:0/3853232639 wait complete. 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.733+0000 7f959de2f740 1 -- start start 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 Processor -- start 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8673d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 0x55c7b86702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941b0330 con 0x55c7b86639e0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941afcd0 con 0x55c7b8662810 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941ad1d0 con 0x55c7b86643a0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f959de2f740 10 client.0 _ll_register_callbacks cb 0x7f9584004880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9593400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8673d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9599000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 0x55c7b86702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9593e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56026/0 (socket says 172.21.15.99:56026) 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9599000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 0x55c7b86702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48608/0 (socket says 172.21.15.99:48608) 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.734+0000 7f9593e00640 1 -- 172.21.15.99:0/847774801 learned_addr learned my addr 172.21.15.99:0/847774801 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 -- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 msgr2=0x55c7b86702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 0x55c7b86702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 -- 172.21.15.99:0/847774801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 msgr2=0x55c7b8673d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8673d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 -- 172.21.15.99:0/847774801 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7b8670920 con 0x55c7b8662810 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9593e00640 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95840a3450 tx=0x7f958409ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da35cd889bbdf4ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9582000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840a3980 con 0x55c7b8662810 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f958400e0b0 con 0x55c7b8662810 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.735+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f958409f330 con 0x55c7b8662810 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9584004340 con 0x55c7b8662810 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 10 client.25478 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c7b86d48f0 con 0x55c7b8662810 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f958409f330 con 0x55c7b8662810 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f95840b9ce0 con 0x55c7b8662810 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 10 client.25478 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 10 client.25478 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 20 client.25478 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86d4fb0 0x55c7b86d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 4 client.25478 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c7b85ceb50 con 0x55c7b86d4fb0 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.736+0000 7f9593400640 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86d4fb0 0x55c7b86d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f9593400640 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86d4fb0 0x55c7b86d73a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=243b815bec65df69 server_cookie=48fa078e0a252a80 in_seq=0 out_seq=0 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f9582000640 10 client.25478 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:22.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f9582000640 1 -- 172.21.15.99:0/847774801 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c7b85ceb50 con 0x55c7b86d4fb0 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f9582000640 10 client.25478 handle_command_reply: tid=0 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f959de2f740 1 client.25478 shutdown 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 msgr2=0x55c7b86737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.737+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95840a3450 tx=0x7f958409ba50 comp rx=0 tx=0).stop 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 shutdown_connections 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b86643a0 0x55c7b86702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c7b86d4fb0 0x55c7b86d73a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b86639e0 0x55c7b8673d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 --2- 172.21.15.99:0/847774801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7b8662810 0x55c7b86737c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 >> 172.21.15.99:0/847774801 conn(0x55c7b865dc00 msgr2=0x55c7b8666c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f9582000640 0 client.25478 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 shutdown_connections 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 1 -- 172.21.15.99:0/847774801 wait complete. 2024-03-23T23:44:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.738+0000 7f959de2f740 20 client.25478 trim_cache size 0 max 16384 2024-03-23T23:44:22.744 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 msgr2=0x7f95780a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 0x7f95780a4c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f958c009ea0 tx=0x7f958c009ca0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 msgr2=0x7f959419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f958400c420 tx=0x7f9584009f90 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 shutdown_connections 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f95780a2790 0x7f95780a4c50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9594077c40 0x7f95941a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940772a0 0x7f959419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 --2- 172.21.15.99:0/2918655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9594076060 0x7f959419dfa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 >> 172.21.15.99:0/2918655891 conn(0x7f95940fee50 msgr2=0x7f9594100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 shutdown_connections 2024-03-23T23:44:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.739+0000 7f959b000640 1 -- 172.21.15.99:0/2918655891 wait complete. 2024-03-23T23:44:22.757 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:22.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T23:44:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 -- 172.21.15.99:0/3067296096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 msgr2=0x7ff76c106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 --2- 172.21.15.99:0/3067296096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c106d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff754009900 tx=0x7ff75402bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 -- 172.21.15.99:0/3067296096 shutdown_connections 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 --2- 172.21.15.99:0/3067296096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 --2- 172.21.15.99:0/3067296096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c106d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 --2- 172.21.15.99:0/3067296096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 0x7ff76c104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 -- 172.21.15.99:0/3067296096 >> 172.21.15.99:0/3067296096 conn(0x7ff76c0fba10 msgr2=0x7ff76c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 -- 172.21.15.99:0/3067296096 shutdown_connections 2024-03-23T23:44:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.992+0000 7ff771a00640 1 -- 172.21.15.99:0/3067296096 wait complete. 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.993+0000 7ff771a00640 1 Processor -- start 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.993+0000 7ff771a00640 1 -- start start 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 0x7ff76c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:22.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c10d250 con 0x7ff76c102020 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c10d0d0 con 0x7ff76c104950 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c10d3d0 con 0x7ff76c107280 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56056/0 (socket says 172.21.15.99:56056) 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 -- 172.21.15.99:0/2142855706 learned_addr learned my addr 172.21.15.99:0/2142855706 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff76be00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 0x7ff76c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771000640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 -- 172.21.15.99:0/2142855706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 msgr2=0x7ff76c100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 -- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 msgr2=0x7ff76c101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 0x7ff76c101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff763e00640 1 -- 172.21.15.99:0/2142855706 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c100da0 con 0x7ff76c104950 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.994+0000 7ff771000640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff763e00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff754000c00 tx=0x7ff75402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:22.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff769600640 1 -- 172.21.15.99:0/2142855706 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754043020 con 0x7ff76c104950 2024-03-23T23:44:22.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff769600640 1 -- 172.21.15.99:0/2142855706 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7540054e0 con 0x7ff76c104950 2024-03-23T23:44:22.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c1a19e0 con 0x7ff76c104950 2024-03-23T23:44:22.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff769600640 1 -- 172.21.15.99:0/2142855706 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754003c40 con 0x7ff76c104950 2024-03-23T23:44:22.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.995+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c1a1f80 con 0x7ff76c104950 2024-03-23T23:44:22.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:22.999+0000 7ff775d8a740 1 Processor -- start 2024-03-23T23:44:23.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff769600640 1 -- 172.21.15.99:0/2142855706 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff754046020 con 0x7ff76c104950 2024-03-23T23:44:23.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 -- start start 2024-03-23T23:44:23.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ac9c2c0 con 0x55af0ad379f0 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ac44c90 con 0x55af0ad38bc0 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff775d8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ac98360 con 0x55af0ad39580 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff771000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff76be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff76be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56064/0 (socket says 172.21.15.99:56064) 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff771000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39938/0 (socket says 172.21.15.99:39938) 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff76be00640 1 -- 172.21.15.99:0/4130377390 learned_addr learned my addr 172.21.15.99:0/4130377390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.000+0000 7ff763e00640 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 -- 172.21.15.99:0/4130377390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 msgr2=0x55af0ad3fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad3fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 -- 172.21.15.99:0/4130377390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 msgr2=0x55af0ad37e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 -- 172.21.15.99:0/4130377390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0ac69080 con 0x55af0ad38bc0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff771000640 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff76be00640 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009cf0 tx=0x7ff75c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc78e52b9860a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff761600640 1 -- 172.21.15.99:0/4130377390 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c004230 con 0x55af0ad38bc0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff761600640 1 -- 172.21.15.99:0/4130377390 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75c0043d0 con 0x55af0ad38bc0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.001+0000 7ff761600640 1 -- 172.21.15.99:0/4130377390 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c019960 con 0x55af0ad38bc0 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff769600640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 0x7ff7500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff769600640 1 -- 172.21.15.99:0/2142855706 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff7540f0a10 con 0x7ff76c104950 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff771000640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 0x7ff7500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 -- 172.21.15.99:0/4130377390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 msgr2=0x55af0ad39040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009cf0 tx=0x7ff75c011810 comp rx=0 tx=0).stop 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 -- 172.21.15.99:0/4130377390 shutdown_connections 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad39040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 --2- 172.21.15.99:0/4130377390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad379f0 0x55af0ad37e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.002+0000 7ff775d8a740 1 -- 172.21.15.99:0/4130377390 >> 172.21.15.99:0/4130377390 conn(0x55af0ad32de0 msgr2=0x55af0ad331e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.003+0000 7ff775d8a740 1 -- 172.21.15.99:0/4130377390 shutdown_connections 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.003+0000 7ff771000640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 0x7ff7500a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55af0ad38ac0 tx=0x7ff764009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.003+0000 7ff775d8a740 1 -- 172.21.15.99:0/4130377390 wait complete. 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.003+0000 7ff775d8a740 1 -- start start 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.003+0000 7ff775d8a740 1 Processor -- start 2024-03-23T23:44:23.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ad40590 con 0x55af0ad38bc0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ac580d0 con 0x55af0ad379f0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ad40710 con 0x55af0ad39580 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff775d8a740 10 client.0 _ll_register_callbacks cb 0x55af0ac5ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff763e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff771000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56066/0 (socket says 172.21.15.99:56066) 2024-03-23T23:44:23.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 -- 172.21.15.99:0/529079560 learned_addr learned my addr 172.21.15.99:0/529079560 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 -- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 msgr2=0x55af0ad491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 -- 172.21.15.99:0/529079560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 msgr2=0x55af0ad421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff76be00640 1 -- 172.21.15.99:0/529079560 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af0a9b03f0 con 0x55af0ad379f0 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.004+0000 7ff771000640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff76be00640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff75c004630 tx=0x7ff75c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7d2b44935644a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff75a000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c027070 con 0x55af0ad379f0 2024-03-23T23:44:23.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff75c005240 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75c0043d0 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c033050 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff775d8a740 10 client.25490 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.005+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af0aa7a7d0 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75c0043d0 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff75c004ba0 con 0x55af0ad379f0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 10 client.25490 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 10 client.25490 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 20 client.25490 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ad4d410 0x55af0ad4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 4 client.25490 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af0aca9800 con 0x55af0ad4d410 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.006+0000 7ff763e00640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ad4d410 0x55af0ad4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.007+0000 7ff763e00640 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ad4d410 0x55af0ad4f860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36748c72618febc1 server_cookie=97b487073b4a45d3 in_seq=0 out_seq=0 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.007+0000 7ff75a000640 10 client.25490 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff75a000640 1 -- 172.21.15.99:0/529079560 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55af0aca9800 con 0x55af0ad4d410 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff75a000640 10 client.25490 handle_command_reply: tid=0 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 client.25490 shutdown 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 msgr2=0x55af0ad41c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff75c004630 tx=0x7ff75c003820 comp rx=0 tx=0).stop 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 shutdown_connections 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ad4d410 0x55af0ad4f860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad39580 0x55af0ad491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 --2- 172.21.15.99:0/529079560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad379f0 0x55af0ad41c60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 >> 172.21.15.99:0/529079560 conn(0x55af0ad32de0 msgr2=0x55af0ad32160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff75a000640 0 client.25490 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 shutdown_connections 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.008+0000 7ff775d8a740 1 -- 172.21.15.99:0/529079560 wait complete. 2024-03-23T23:44:23.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.009+0000 7ff775d8a740 20 client.25490 trim_cache size 0 max 16384 2024-03-23T23:44:23.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 Processor -- start 2024-03-23T23:44:23.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 -- start start 2024-03-23T23:44:23.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad3c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad39580 0x55af0ad3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1aaa80 con 0x55af0ad38bc0 2024-03-23T23:44:23.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1ac400 con 0x55af0ad39580 2024-03-23T23:44:23.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.022+0000 7ff775d8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1ab0e0 con 0x55af0ad379f0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff763e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad3c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48660/0 (socket says 172.21.15.99:48660) 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 -- 172.21.15.99:0/2434796447 learned_addr learned my addr 172.21.15.99:0/2434796447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff771000640 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad39580 0x55af0ad3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 -- 172.21.15.99:0/2434796447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad39580 msgr2=0x55af0ad3c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad39580 0x55af0ad3c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 -- 172.21.15.99:0/2434796447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 msgr2=0x55af0ad3c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad3c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 -- 172.21.15.99:0/2434796447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0ad38640 con 0x55af0ad379f0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff763e00640 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.023+0000 7ff76be00640 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0059f0 tx=0x7ff75c009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace213b4a971ce33 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff760c00640 1 -- 172.21.15.99:0/2434796447 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c027070 con 0x55af0ad379f0 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff760c00640 1 -- 172.21.15.99:0/2434796447 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75c003de0 con 0x55af0ad379f0 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff760c00640 1 -- 172.21.15.99:0/2434796447 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c019420 con 0x55af0ad379f0 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 -- 172.21.15.99:0/2434796447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 msgr2=0x55af0ad3bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0059f0 tx=0x7ff75c009b70 comp rx=0 tx=0).stop 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 -- 172.21.15.99:0/2434796447 shutdown_connections 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad39580 0x55af0ad3c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad38bc0 0x55af0ad3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2434796447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad3bc60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 -- 172.21.15.99:0/2434796447 >> 172.21.15.99:0/2434796447 conn(0x55af0ad32de0 msgr2=0x55af0ad368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 -- 172.21.15.99:0/2434796447 shutdown_connections 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.024+0000 7ff775d8a740 1 -- 172.21.15.99:0/2434796447 wait complete. 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 -- start start 2024-03-23T23:44:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 Processor -- start 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 0x55af0ad47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1abda0 con 0x55af0ad39580 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1ab740 con 0x55af0ad38bc0 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c1a8c40 con 0x55af0ad379f0 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff775d8a740 10 client.0 _ll_register_callbacks cb 0x7ff764009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff763e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.025+0000 7ff76be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48668/0 (socket says 172.21.15.99:48668) 2024-03-23T23:44:23.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff763e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56092/0 (socket says 172.21.15.99:56092) 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 -- 172.21.15.99:0/2799344790 learned_addr learned my addr 172.21.15.99:0/2799344790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff771000640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 0x55af0ad47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 -- 172.21.15.99:0/2799344790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 msgr2=0x55af0ad44580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 -- 172.21.15.99:0/2799344790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 msgr2=0x55af0ad47550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 0x55af0ad47550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 -- 172.21.15.99:0/2799344790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af0ad47b40 con 0x55af0ad379f0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff763e00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff771000640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 0x55af0ad47550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff76be00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff75c003fd0 tx=0x7ff75c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f983b8b887a3c88 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff75a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c027070 con 0x55af0ad379f0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75c003de0 con 0x55af0ad379f0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.026+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff75c0097a0 con 0x55af0ad379f0 2024-03-23T23:44:23.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c0197f0 con 0x55af0ad379f0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 10 client.25522 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55af0ada8cf0 con 0x55af0ad379f0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff75c003de0 con 0x55af0ad379f0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff75c004630 con 0x55af0ad379f0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 10 client.25522 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 10 client.25522 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 20 client.25522 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ada9530 0x55af0adab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 4 client.25522 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af0aca3b50 con 0x55af0ada9530 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.027+0000 7ff763e00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ada9530 0x55af0adab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff75a000640 10 client.25522 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff763e00640 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ada9530 0x55af0adab950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4ae73348f6148188 server_cookie=be3ff28b379c4f2a in_seq=0 out_seq=0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff75a000640 1 -- 172.21.15.99:0/2799344790 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af0aca3b50 con 0x55af0ada9530 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff75a000640 10 client.25522 handle_command_reply: tid=0 2024-03-23T23:44:23.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 client.25522 shutdown 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 msgr2=0x55af0ad44040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff75c003fd0 tx=0x7ff75c00ee60 comp rx=0 tx=0).stop 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 shutdown_connections 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55af0ada9530 0x55af0adab950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55af0ad39580 0x55af0ad47550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af0ad38bc0 0x55af0ad44580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 --2- 172.21.15.99:0/2799344790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af0ad379f0 0x55af0ad44040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff75a000640 0 client.25522 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.028+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 >> 172.21.15.99:0/2799344790 conn(0x55af0ad32de0 msgr2=0x55af0ad318d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 shutdown_connections 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff775d8a740 1 -- 172.21.15.99:0/2799344790 wait complete. 2024-03-23T23:44:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff775d8a740 20 client.25522 trim_cache size 0 max 16384 2024-03-23T23:44:23.034 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 msgr2=0x7ff7500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 0x7ff7500a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55af0ad38ac0 tx=0x7ff764009500 comp rx=0 tx=0).stop 2024-03-23T23:44:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 msgr2=0x7ff76c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff754000c00 tx=0x7ff75402c650 comp rx=0 tx=0).stop 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.029+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 shutdown_connections 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff7500a2880 0x7ff7500a4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c107280 0x7ff76c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff76c104950 0x7ff76c0fffd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 --2- 172.21.15.99:0/2142855706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff76c102020 0x7ff76c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 >> 172.21.15.99:0/2142855706 conn(0x7ff76c0fba10 msgr2=0x7ff76c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 shutdown_connections 2024-03-23T23:44:23.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.030+0000 7ff771a00640 1 -- 172.21.15.99:0/2142855706 wait complete. 2024-03-23T23:44:23.055 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:23.055 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T23:44:23.110 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:22 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2212344183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:23.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:22 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2212344183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:23.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:22 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2212344183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 -- 172.21.15.99:0/340988810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068102790 msgr2=0x7f0068102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 --2- 172.21.15.99:0/340988810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068102790 0x7f0068102c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0058009900 tx=0x7f005802bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 -- 172.21.15.99:0/340988810 shutdown_connections 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 --2- 172.21.15.99:0/340988810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0068103150 0x7f00681099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 --2- 172.21.15.99:0/340988810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068102790 0x7f0068102c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 --2- 172.21.15.99:0/340988810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00681015a0 0x7f00681019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 -- 172.21.15.99:0/340988810 >> 172.21.15.99:0/340988810 conn(0x7f00680fcb10 msgr2=0x7f00680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 -- 172.21.15.99:0/340988810 shutdown_connections 2024-03-23T23:44:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.285+0000 7f006f200640 1 -- 172.21.15.99:0/340988810 wait complete. 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.286+0000 7f006f200640 1 Processor -- start 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.286+0000 7f006f200640 1 -- start start 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 0x7f0068073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103150 0x7f0068073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006810d250 con 0x7f00681015a0 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006810d0d0 con 0x7f0068103150 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006810d3d0 con 0x7f0068102790 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39998/0 (socket says 172.21.15.99:39998) 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006de00640 1 -- 172.21.15.99:0/645650627 learned_addr learned my addr 172.21.15.99:0/645650627 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 0x7f0068073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.287+0000 7f006e800640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103150 0x7f0068073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 -- 172.21.15.99:0/645650627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103150 msgr2=0x7f0068073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103150 0x7f0068073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 -- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 msgr2=0x7f0068075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 -- 172.21.15.99:0/645650627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068074860 con 0x7f0068102790 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006de00640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006d400640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 0x7f0068073a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0058036040 tx=0x7f005802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f0062a00640 1 -- 172.21.15.99:0/645650627 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058043020 con 0x7f0068102790 2024-03-23T23:44:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f0062a00640 1 -- 172.21.15.99:0/645650627 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00580054e0 con 0x7f0068102790 2024-03-23T23:44:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f0062a00640 1 -- 172.21.15.99:0/645650627 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058003c10 con 0x7f0068102790 2024-03-23T23:44:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.288+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00681a1950 con 0x7f0068102790 2024-03-23T23:44:23.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.289+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681a1e20 con 0x7f0068102790 2024-03-23T23:44:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.291+0000 7f0062a00640 1 -- 172.21.15.99:0/645650627 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0058046020 con 0x7f0068102790 2024-03-23T23:44:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 Processor -- start 2024-03-23T23:44:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 -- start start 2024-03-23T23:44:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924d96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924d9ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924cfb2c0 con 0x558924d98580 2024-03-23T23:44:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924ca3c90 con 0x558924d969f0 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f0073571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924cf7360 con 0x558924d97bc0 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.292+0000 7f006e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924d96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924d96e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56120/0 (socket says 172.21.15.99:56120) 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48690/0 (socket says 172.21.15.99:48690) 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006e800640 1 -- 172.21.15.99:0/3692083096 learned_addr learned my addr 172.21.15.99:0/3692083096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006d400640 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924d9ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 -- 172.21.15.99:0/3692083096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 msgr2=0x558924d96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924d96e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 -- 172.21.15.99:0/3692083096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 msgr2=0x558924d9ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924d9ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 -- 172.21.15.99:0/3692083096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558924cc8080 con 0x558924d97bc0 2024-03-23T23:44:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006d400640 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924d9ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f0062a00640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 0x7f00500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f006de00640 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f005c0114c0 tx=0x7f005c011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=756e99520a6f0867 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.293+0000 7f0062a00640 1 -- 172.21.15.99:0/645650627 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f00580f0a10 con 0x7f0068102790 2024-03-23T23:44:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f006e800640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 0x7f00500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0057000640 1 -- 172.21.15.99:0/3692083096 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c005020 con 0x558924d97bc0 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0057000640 1 -- 172.21.15.99:0/3692083096 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f005c0047a0 con 0x558924d97bc0 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0057000640 1 -- 172.21.15.99:0/3692083096 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c00f9d0 con 0x558924d97bc0 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 -- 172.21.15.99:0/3692083096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 msgr2=0x558924d98040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f005c0114c0 tx=0x7f005c011980 comp rx=0 tx=0).stop 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f006e800640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 0x7f00500a4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x558924d97ac0 tx=0x7f0064009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 -- 172.21.15.99:0/3692083096 shutdown_connections 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924d9ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d98040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 --2- 172.21.15.99:0/3692083096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924d96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.294+0000 7f0073571740 1 -- 172.21.15.99:0/3692083096 >> 172.21.15.99:0/3692083096 conn(0x558924d91de0 msgr2=0x558924d921e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.295+0000 7f0073571740 1 -- 172.21.15.99:0/3692083096 shutdown_connections 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.295+0000 7f0073571740 1 -- 172.21.15.99:0/3692083096 wait complete. 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.295+0000 7f0073571740 1 -- start start 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.295+0000 7f0073571740 1 Processor -- start 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924da81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924d9f590 con 0x558924d98580 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924cb70d0 con 0x558924d97bc0 2024-03-23T23:44:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558924d9f710 con 0x558924d969f0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f0073571740 10 client.0 _ll_register_callbacks cb 0x558924cbbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48700/0 (socket says 172.21.15.99:48700) 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006de00640 1 -- 172.21.15.99:0/74471599 learned_addr learned my addr 172.21.15.99:0/74471599 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006e800640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924da81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 -- 172.21.15.99:0/74471599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 msgr2=0x558924da0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 -- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 msgr2=0x558924da81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924da81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.296+0000 7f006d400640 1 -- 172.21.15.99:0/74471599 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558924a0f3f0 con 0x558924d97bc0 2024-03-23T23:44:23.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f006d400640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da1180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0058009900 tx=0x7f00580bd100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce9b98ed5873304 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00580035f0 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0058003db0 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f005c0047a0 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0058005690 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f0073571740 10 client.25508 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558924ad97d0 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f005c0047a0 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.297+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f00580c1100 con 0x558924d97bc0 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 10 client.25508 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 10 client.25508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 20 client.25508 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924dac580 0x558924daea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 4 client.25508 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:23.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558924d08800 con 0x558924dac580 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.298+0000 7f006de00640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924dac580 0x558924daea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.299+0000 7f004b400640 10 client.25508 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.299+0000 7f006de00640 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924dac580 0x558924daea20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c26f0a5f066af464 server_cookie=ab054d0da5f8eb04 in_seq=0 out_seq=0 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.299+0000 7f004b400640 1 -- 172.21.15.99:0/74471599 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558924d08800 con 0x558924dac580 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.299+0000 7f004b400640 10 client.25508 handle_command_reply: tid=0 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.299+0000 7f0073571740 1 client.25508 shutdown 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 msgr2=0x558924da1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da1180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0058009900 tx=0x7f00580bd100 comp rx=0 tx=0).stop 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 shutdown_connections 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924dac580 0x558924daea20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d98580 0x558924da81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da1180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f004b400640 0 client.25508 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 --2- 172.21.15.99:0/74471599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 >> 172.21.15.99:0/74471599 conn(0x558924d91de0 msgr2=0x558924d91160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 shutdown_connections 2024-03-23T23:44:23.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 1 -- 172.21.15.99:0/74471599 wait complete. 2024-03-23T23:44:23.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.300+0000 7f0073571740 20 client.25508 trim_cache size 0 max 16384 2024-03-23T23:44:23.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.313+0000 7f0073571740 1 Processor -- start 2024-03-23T23:44:23.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.313+0000 7f0073571740 1 -- start start 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924d9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924da8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681aa960 con 0x558924d91de0 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681ac2e0 con 0x558924d969f0 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f0073571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681aafc0 con 0x558924d97bc0 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924d9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924da8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d9ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48706/0 (socket says 172.21.15.99:48706) 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006e800640 1 -- 172.21.15.99:0/2197181538 learned_addr learned my addr 172.21.15.99:0/2197181538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 -- 172.21.15.99:0/2197181538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 msgr2=0x558924d9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d9ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 -- 172.21.15.99:0/2197181538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 msgr2=0x558924d9eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924d9eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.314+0000 7f006d400640 1 -- 172.21.15.99:0/2197181538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558924d97640 con 0x558924d969f0 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f006d400640 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924da8180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f005802c7b0 tx=0x7f00580bb840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f71d67037c991e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f0056600640 1 -- 172.21.15.99:0/2197181538 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00580035f0 con 0x558924d969f0 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f0056600640 1 -- 172.21.15.99:0/2197181538 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00580c1480 con 0x558924d969f0 2024-03-23T23:44:23.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f0056600640 1 -- 172.21.15.99:0/2197181538 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058003e40 con 0x558924d969f0 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f0073571740 1 -- 172.21.15.99:0/2197181538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 msgr2=0x558924da8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.315+0000 7f0073571740 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924da8180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f005802c7b0 tx=0x7f00580bb840 comp rx=0 tx=0).stop 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 -- 172.21.15.99:0/2197181538 shutdown_connections 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d97bc0 0x558924d9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d969f0 0x558924da8180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 --2- 172.21.15.99:0/2197181538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924d9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 -- 172.21.15.99:0/2197181538 >> 172.21.15.99:0/2197181538 conn(0x558924d9fe50 msgr2=0x558924d958d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 -- 172.21.15.99:0/2197181538 shutdown_connections 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 -- 172.21.15.99:0/2197181538 wait complete. 2024-03-23T23:44:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 -- start start 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.316+0000 7f0073571740 1 Processor -- start 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681abc80 con 0x558924d91de0 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681ab620 con 0x558924d97bc0 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a8b20 con 0x558924d969f0 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f0073571740 10 client.0 _ll_register_callbacks cb 0x7f0058004310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40036/0 (socket says 172.21.15.99:40036) 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006de00640 1 -- 172.21.15.99:0/875822668 learned_addr learned my addr 172.21.15.99:0/875822668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 -- 172.21.15.99:0/875822668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 msgr2=0x558924da76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 -- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 msgr2=0x558924da4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 -- 172.21.15.99:0/875822668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558924da7ca0 con 0x558924d969f0 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006de00640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f006d400640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da4690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f00580bd2c0 tx=0x7f00580f6260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738b4d61aee10eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.317+0000 7f004b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:23.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00580dd2e0 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f00580bc780 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558924d02b50 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00580035f0 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 10 client.25540 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558924da7ca0 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558924d02b50 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f00580bc1f0 con 0x558924d969f0 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 10 client.25540 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 10 client.25540 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 20 client.25540 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924d98580 0x558924e0ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 4 client.25540 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.318+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558924c80500 con 0x558924d98580 2024-03-23T23:44:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.319+0000 7f006de00640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924d98580 0x558924e0ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.319+0000 7f006de00640 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924d98580 0x558924e0ac10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ad0deeee276844f server_cookie=7001f825628ea67a in_seq=0 out_seq=0 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f004b400640 10 client.25540 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f004b400640 1 -- 172.21.15.99:0/875822668 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558924c80500 con 0x558924d98580 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f004b400640 10 client.25540 handle_command_reply: tid=0 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 client.25540 shutdown 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 msgr2=0x558924da4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da4690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f00580bd2c0 tx=0x7f00580f6260 comp rx=0 tx=0).stop 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 shutdown_connections 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x558924d98580 0x558924e0ac10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558924d97bc0 0x558924da76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558924d969f0 0x558924da4690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 --2- 172.21.15.99:0/875822668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558924d91de0 0x558924da4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 >> 172.21.15.99:0/875822668 conn(0x558924d9fe50 msgr2=0x558924d958d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f004b400640 0 client.25540 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:23.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 shutdown_connections 2024-03-23T23:44:23.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 1 -- 172.21.15.99:0/875822668 wait complete. 2024-03-23T23:44:23.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.320+0000 7f0073571740 20 client.25540 trim_cache size 0 max 16384 2024-03-23T23:44:23.326 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:23.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 msgr2=0x7f00500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 0x7f00500a4d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x558924d97ac0 tx=0x7f0064009500 comp rx=0 tx=0).stop 2024-03-23T23:44:23.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 msgr2=0x7f0068073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 0x7f0068073a90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0058036040 tx=0x7f005802c650 comp rx=0 tx=0).stop 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 shutdown_connections 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00500a2880 0x7f00500a4d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103150 0x7f0068073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0068102790 0x7f0068073a90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 --2- 172.21.15.99:0/645650627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00681015a0 0x7f0068075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 >> 172.21.15.99:0/645650627 conn(0x7f00680fcb10 msgr2=0x7f0068105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 shutdown_connections 2024-03-23T23:44:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.321+0000 7f006f200640 1 -- 172.21.15.99:0/645650627 wait complete. 2024-03-23T23:44:23.337 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:23.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 -- 172.21.15.99:0/3574617099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4107390 msgr2=0x7f3fd41097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/3574617099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4107390 0x7f3fd41097e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009ca0 tx=0x7f3fcc02c0f0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 -- 172.21.15.99:0/3574617099 shutdown_connections 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/3574617099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4107390 0x7f3fd41097e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/3574617099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4104a60 0x7f3fd4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/3574617099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4102130 0x7f3fd4104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 -- 172.21.15.99:0/3574617099 >> 172.21.15.99:0/3574617099 conn(0x7f3fd40fbb40 msgr2=0x7f3fd40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 -- 172.21.15.99:0/3574617099 shutdown_connections 2024-03-23T23:44:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.544+0000 7f3fd8c00640 1 -- 172.21.15.99:0/3574617099 wait complete. 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 Processor -- start 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 -- start start 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4102130 0x7f3fd4101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4107390 0x7f3fd4100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd410d330 con 0x7f3fd4102130 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd410d1b0 con 0x7f3fd4107390 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.546+0000 7f3fd8c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd410d4b0 con 0x7f3fd4104a60 2024-03-23T23:44:23.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48724/0 (socket says 172.21.15.99:48724) 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 -- 172.21.15.99:0/1030126766 learned_addr learned my addr 172.21.15.99:0/1030126766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4102130 0x7f3fd4101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4107390 0x7f3fd4100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 -- 172.21.15.99:0/1030126766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4107390 msgr2=0x7f3fd4100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4107390 0x7f3fd4100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 -- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4102130 msgr2=0x7f3fd4101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4102130 0x7f3fd4101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 -- 172.21.15.99:0/1030126766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4100eb0 con 0x7f3fd4104a60 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.547+0000 7f3fd1400640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9a0 tx=0x7f3fc000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.548+0000 7f3fcaa00640 1 -- 172.21.15.99:0/1030126766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000cd80 con 0x7f3fd4104a60 2024-03-23T23:44:23.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.548+0000 7f3fcaa00640 1 -- 172.21.15.99:0/1030126766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc00035f0 con 0x7f3fd4104a60 2024-03-23T23:44:23.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.548+0000 7f3fcaa00640 1 -- 172.21.15.99:0/1030126766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0010670 con 0x7f3fd4104a60 2024-03-23T23:44:23.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.548+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41a1ac0 con 0x7f3fd4104a60 2024-03-23T23:44:23.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.548+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd41a2060 con 0x7f3fd4104a60 2024-03-23T23:44:23.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.551+0000 7f3fdb83b740 1 Processor -- start 2024-03-23T23:44:23.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.551+0000 7f3fdb83b740 1 -- start start 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.551+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.551+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd949e0 0x55df6cd94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd953a0 0x55df6cd9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6ccf82c0 con 0x55df6cd953a0 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6cca0c90 con 0x55df6cd93810 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6ccf4360 con 0x55df6cd949e0 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56174/0 (socket says 172.21.15.99:56174) 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 -- 172.21.15.99:0/1622037934 learned_addr learned my addr 172.21.15.99:0/1622037934 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd949e0 0x55df6cd94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 -- 172.21.15.99:0/1622037934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd949e0 msgr2=0x55df6cd94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd949e0 0x55df6cd94e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 -- 172.21.15.99:0/1622037934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd953a0 msgr2=0x55df6cd9bc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd953a0 0x55df6cd9bc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 -- 172.21.15.99:0/1622037934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df6ccc5080 con 0x55df6cd93810 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fcaa00640 1 -- 172.21.15.99:0/1030126766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3fc00037a0 con 0x7f3fd4104a60 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.552+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc02c700 tx=0x7f3fcc0323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e867f4e4606ab37 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.553+0000 7f3fbf000640 1 -- 172.21.15.99:0/1622037934 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc043070 con 0x55df6cd93810 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.553+0000 7f3fbf000640 1 -- 172.21.15.99:0/1622037934 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc02cca0 con 0x55df6cd93810 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.553+0000 7f3fbf000640 1 -- 172.21.15.99:0/1622037934 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc039040 con 0x55df6cd93810 2024-03-23T23:44:23.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.553+0000 7f3fdb83b740 1 -- 172.21.15.99:0/1622037934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 msgr2=0x55df6cd93c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.553+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc02c700 tx=0x7f3fcc0323d0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 -- 172.21.15.99:0/1622037934 shutdown_connections 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd953a0 0x55df6cd9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd949e0 0x55df6cd94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/1622037934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd93810 0x55df6cd93c30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 -- 172.21.15.99:0/1622037934 >> 172.21.15.99:0/1622037934 conn(0x55df6cd8ec00 msgr2=0x55df6cd8f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fcaa00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 0x7f3fb80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 -- 172.21.15.99:0/1622037934 shutdown_connections 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 -- 172.21.15.99:0/1622037934 wait complete. 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fcaa00640 1 -- 172.21.15.99:0/1030126766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fc00cecb0 con 0x7f3fd4104a60 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 0x7f3fb80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.554+0000 7f3fdb83b740 1 -- start start 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 Processor -- start 2024-03-23T23:44:23.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fd2800640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 0x7f3fb80a4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009bf0 tx=0x7f3fcc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6cd9c350 con 0x55df6cd93810 2024-03-23T23:44:23.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6ccb40d0 con 0x55df6cd949e0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6cd9c4d0 con 0x55df6cd953a0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fdb83b740 10 client.0 _ll_register_callbacks cb 0x55df6ccb8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.555+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56186/0 (socket says 172.21.15.99:56186) 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 -- 172.21.15.99:0/2574841724 learned_addr learned my addr 172.21.15.99:0/2574841724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 -- 172.21.15.99:0/2574841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 msgr2=0x55df6cda4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 -- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 msgr2=0x55df6cd9da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd9da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 -- 172.21.15.99:0/2574841724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df6ca0c3f0 con 0x55df6cd949e0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3fc009b5c0 tx=0x7f3fc009b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedfa30f3d193406 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc009f780 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fc0084bc0 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.556+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fcc02cca0 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc00a3dd0 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 10 client.25526 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55df6cad67d0 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc02cca0 con 0x55df6cd949e0 2024-03-23T23:44:23.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3fc009aa80 con 0x55df6cd949e0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 10 client.25526 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 10 client.25526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 20 client.25526 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6cda94a0 0x55df6cdab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 4 client.25526 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.557+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df6cd05800 con 0x55df6cda94a0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.558+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6cda94a0 0x55df6cdab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.558+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6cda94a0 0x55df6cdab8c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0a3c8915efe3788 server_cookie=4adb633be733b6ba in_seq=0 out_seq=0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.558+0000 7f3fb3400640 10 client.25526 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fb3400640 1 -- 172.21.15.99:0/2574841724 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df6cd05800 con 0x55df6cda94a0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fb3400640 10 client.25526 handle_command_reply: tid=0 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 client.25526 shutdown 2024-03-23T23:44:23.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 msgr2=0x55df6cd9df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3fc009b5c0 tx=0x7f3fc009b050 comp rx=0 tx=0).stop 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 shutdown_connections 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6cda94a0 0x55df6cdab8c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cd9df50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fb3400640 0 client.25526 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2574841724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd9da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.559+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 >> 172.21.15.99:0/2574841724 conn(0x55df6cd8ec00 msgr2=0x55df6cd8deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.560+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 shutdown_connections 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.560+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2574841724 wait complete. 2024-03-23T23:44:23.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.560+0000 7f3fdb83b740 20 client.25526 trim_cache size 0 max 16384 2024-03-23T23:44:23.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 Processor -- start 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 -- start start 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd8dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41aac20 con 0x55df6cd93810 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41ac5a0 con 0x55df6cd949e0 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41ab280 con 0x55df6cd953a0 2024-03-23T23:44:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fd2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48754/0 (socket says 172.21.15.99:48754) 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fd2800640 1 -- 172.21.15.99:0/532961738 learned_addr learned my addr 172.21.15.99:0/532961738 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.574+0000 7f3fd1400640 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd8dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 -- 172.21.15.99:0/532961738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 msgr2=0x55df6cda4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 -- 172.21.15.99:0/532961738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 msgr2=0x55df6cd8dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd8dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 -- 172.21.15.99:0/532961738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df6cd9e2c0 con 0x55df6cd953a0 2024-03-23T23:44:23.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fd2800640 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc006010 tx=0x7f3fcc02e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba42c56e027b0ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fbe600640 1 -- 172.21.15.99:0/532961738 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc043040 con 0x55df6cd953a0 2024-03-23T23:44:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fbe600640 1 -- 172.21.15.99:0/532961738 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc04e070 con 0x55df6cd953a0 2024-03-23T23:44:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.575+0000 7f3fbe600640 1 -- 172.21.15.99:0/532961738 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc048910 con 0x55df6cd953a0 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 -- 172.21.15.99:0/532961738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 msgr2=0x55df6cdb0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc006010 tx=0x7f3fcc02e000 comp rx=0 tx=0).stop 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 -- 172.21.15.99:0/532961738 shutdown_connections 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cdb0160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/532961738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cd8dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 -- 172.21.15.99:0/532961738 >> 172.21.15.99:0/532961738 conn(0x55df6cd8ec00 msgr2=0x55df6cd8d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.576+0000 7f3fdb83b740 1 -- 172.21.15.99:0/532961738 shutdown_connections 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.577+0000 7f3fdb83b740 1 -- 172.21.15.99:0/532961738 wait complete. 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.577+0000 7f3fdb83b740 1 -- start start 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.577+0000 7f3fdb83b740 1 Processor -- start 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41abf40 con 0x55df6cd93810 2024-03-23T23:44:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41ab8e0 con 0x55df6cd949e0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41a8de0 con 0x55df6cd953a0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fdb83b740 10 client.0 _ll_register_callbacks cb 0x7f3fc0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fd1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fd1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fd1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40092/0 (socket says 172.21.15.99:40092) 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fd1e00640 1 -- 172.21.15.99:0/2679853953 learned_addr learned my addr 172.21.15.99:0/2679853953 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.578+0000 7f3fd2800640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 -- 172.21.15.99:0/2679853953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 msgr2=0x55df6cda1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 -- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 msgr2=0x55df6cda4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 -- 172.21.15.99:0/2679853953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df6cda50c0 con 0x55df6cd949e0 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd2800640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fd1400640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda1530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00998e0 tx=0x7f3fc009b0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fba4c06c0c253c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fb3400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00041d0 con 0x55df6cd949e0 2024-03-23T23:44:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3fc00b4210 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fcc04e070 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc00a9050 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fdb83b740 10 client.25532 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.579+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55df6ce05c80 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc04e070 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3fc009a4f0 con 0x55df6cd949e0 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 10 client.25532 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 10 client.25532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:44:23.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 20 client.25532 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6ce063a0 0x55df6ce087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 4 client.25532 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df6ccffb50 con 0x55df6ce063a0 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.580+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6ce063a0 0x55df6ce087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fd1e00640 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6ce063a0 0x55df6ce087c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=250002d4b374382d server_cookie=486bcba4186020f in_seq=0 out_seq=0 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fb3400640 10 client.25532 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fb3400640 1 -- 172.21.15.99:0/2679853953 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df6ccffb50 con 0x55df6ce063a0 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fb3400640 10 client.25532 handle_command_reply: tid=0 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 client.25532 shutdown 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 msgr2=0x55df6cda1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda1530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00998e0 tx=0x7f3fc009b0c0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 shutdown_connections 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55df6ce063a0 0x55df6ce087c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6cd953a0 0x55df6cda1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df6cd949e0 0x55df6cda1530 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fb3400640 0 client.25532 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 --2- 172.21.15.99:0/2679853953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df6cd93810 0x55df6cda4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 >> 172.21.15.99:0/2679853953 conn(0x55df6cd8ec00 msgr2=0x55df6cd8deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 shutdown_connections 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.581+0000 7f3fdb83b740 1 -- 172.21.15.99:0/2679853953 wait complete. 2024-03-23T23:44:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fdb83b740 20 client.25532 trim_cache size 0 max 16384 2024-03-23T23:44:23.591 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:23.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 msgr2=0x7f3fb80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 0x7f3fb80a4d40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009bf0 tx=0x7f3fcc004a40 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 msgr2=0x7f3fd41000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9a0 tx=0x7f3fc000ee60 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 shutdown_connections 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.582+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f3fb80a2880 0x7f3fb80a4d40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fd4107390 0x7f3fd4100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4104a60 0x7f3fd41000e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 --2- 172.21.15.99:0/1030126766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd4102130 0x7f3fd4101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 >> 172.21.15.99:0/1030126766 conn(0x7f3fd40fbb40 msgr2=0x7f3fd4102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 shutdown_connections 2024-03-23T23:44:23.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.583+0000 7f3fd8c00640 1 -- 172.21.15.99:0/1030126766 wait complete. 2024-03-23T23:44:23.599 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:23.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T23:44:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 -- 172.21.15.99:0/3994614200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84941016b0 msgr2=0x7f8494101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 --2- 172.21.15.99:0/3994614200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84941016b0 0x7f8494101ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8488009900 tx=0x7f848802bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 -- 172.21.15.99:0/3994614200 shutdown_connections 2024-03-23T23:44:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 --2- 172.21.15.99:0/3994614200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8494103260 0x7f8494109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 --2- 172.21.15.99:0/3994614200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f8494102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 --2- 172.21.15.99:0/3994614200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84941016b0 0x7f8494101ad0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 -- 172.21.15.99:0/3994614200 >> 172.21.15.99:0/3994614200 conn(0x7f84940fcc40 msgr2=0x7f84940ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 -- 172.21.15.99:0/3994614200 shutdown_connections 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.799+0000 7f849c400640 1 -- 172.21.15.99:0/3994614200 wait complete. 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.800+0000 7f849c400640 1 Processor -- start 2024-03-23T23:44:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.800+0000 7f849c400640 1 -- start start 2024-03-23T23:44:23.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8494103260 0x7f84941a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849410d350 con 0x7f84941028a0 2024-03-23T23:44:23.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849410d1d0 con 0x7f8494103260 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849c400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849410d4d0 con 0x7f84941016b0 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f8499a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f8499000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f8499a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48778/0 (socket says 172.21.15.99:48778) 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f8499a00640 1 -- 172.21.15.99:0/3368523602 learned_addr learned my addr 172.21.15.99:0/3368523602 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f8499000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40118/0 (socket says 172.21.15.99:40118) 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.801+0000 7f849a400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8494103260 0x7f84941a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 -- 172.21.15.99:0/3368523602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8494103260 msgr2=0x7f84941a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8494103260 0x7f84941a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 -- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 msgr2=0x7f849419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 -- 172.21.15.99:0/3368523602 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84941a1840 con 0x7f84941016b0 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499000640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8499a00640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f848802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8492000640 1 -- 172.21.15.99:0/3368523602 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488043020 con 0x7f84941016b0 2024-03-23T23:44:23.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8492000640 1 -- 172.21.15.99:0/3368523602 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84880054e0 con 0x7f84941016b0 2024-03-23T23:44:23.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f8492000640 1 -- 172.21.15.99:0/3368523602 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488003cd0 con 0x7f84941016b0 2024-03-23T23:44:23.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84941a1ad0 con 0x7f84941016b0 2024-03-23T23:44:23.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.802+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84941a1fd0 con 0x7f84941016b0 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.805+0000 7f849f16a740 1 Processor -- start 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f8492000640 1 -- 172.21.15.99:0/3368523602 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8488046020 con 0x7f84941016b0 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 -- start start 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 0x55a07ded2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded39e0 0x55a07ded3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07de372c0 con 0x55a07ded39e0 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07dddfc90 con 0x55a07ded2810 2024-03-23T23:44:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849f16a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07de33360 con 0x55a07ded43a0 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f8499000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f849a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 0x55a07ded2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f8499000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48780/0 (socket says 172.21.15.99:48780) 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f8499000640 1 -- 172.21.15.99:0/3301032319 learned_addr learned my addr 172.21.15.99:0/3301032319 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.806+0000 7f8499a00640 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded39e0 0x55a07ded3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 -- 172.21.15.99:0/3301032319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 msgr2=0x55a07ded2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 0x55a07ded2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 -- 172.21.15.99:0/3301032319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded39e0 msgr2=0x55a07ded3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded39e0 0x55a07ded3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 -- 172.21.15.99:0/3301032319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07de04080 con 0x55a07ded43a0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f849a400640 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 0x55a07ded2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8499000640 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84800115f0 tx=0x7f8480011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1029b1f8024b85d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8486a00640 1 -- 172.21.15.99:0/3301032319 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848001e070 con 0x55a07ded43a0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8486a00640 1 -- 172.21.15.99:0/3301032319 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f848000fb20 con 0x55a07ded43a0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8492000640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 0x7f847c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.807+0000 7f8486a00640 1 -- 172.21.15.99:0/3301032319 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480019c40 con 0x55a07ded43a0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f8492000640 1 -- 172.21.15.99:0/3368523602 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f84880f0ad0 con 0x7f84941016b0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849a400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 0x7f847c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 -- 172.21.15.99:0/3301032319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 msgr2=0x55a07dedac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84800115f0 tx=0x7f8480011ab0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 -- 172.21.15.99:0/3301032319 shutdown_connections 2024-03-23T23:44:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dedac30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded39e0 0x55a07ded3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 --2- 172.21.15.99:0/3301032319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded2810 0x55a07ded2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849f16a740 1 -- 172.21.15.99:0/3301032319 >> 172.21.15.99:0/3301032319 conn(0x55a07decdc00 msgr2=0x55a07dece000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.808+0000 7f849a400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 0x7f847c0a4d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55a07ded38e0 tx=0x7f848c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.809+0000 7f849f16a740 1 -- 172.21.15.99:0/3301032319 shutdown_connections 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.809+0000 7f849f16a740 1 -- 172.21.15.99:0/3301032319 wait complete. 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.809+0000 7f849f16a740 1 -- start start 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.809+0000 7f849f16a740 1 Processor -- start 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dee4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07dedb3b0 con 0x55a07ded2810 2024-03-23T23:44:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07ddf30d0 con 0x55a07ded39e0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07dedb530 con 0x55a07ded43a0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849f16a740 10 client.0 _ll_register_callbacks cb 0x55a07ddf7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56256/0 (socket says 172.21.15.99:56256) 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 -- 172.21.15.99:0/1899508461 learned_addr learned my addr 172.21.15.99:0/1899508461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f849a400640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dee4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 -- 172.21.15.99:0/1899508461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 msgr2=0x55a07dee4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dee4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 -- 172.21.15.99:0/1899508461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 msgr2=0x55a07dedcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499000640 1 -- 172.21.15.99:0/1899508461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a07db4b3f0 con 0x55a07ded39e0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.810+0000 7f8499a00640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f8499000640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8480005890 tx=0x7f848000f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5cfabd132f7dea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f847b400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480027060 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f84800199c0 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f848000fb20 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f848001e040 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f849f16a740 10 client.25544 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.811+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a07dc157d0 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f848000fb20 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8480021070 con 0x55a07ded39e0 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 10 client.25544 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 20 client.25544 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07dee8190 0x55a07deea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 4 client.25544 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.812+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a07de44800 con 0x55a07dee8190 2024-03-23T23:44:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.813+0000 7f8499a00640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07dee8190 0x55a07deea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.813+0000 7f8499a00640 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07dee8190 0x55a07deea580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2375312eb80baa9c server_cookie=a159574edb831ae2 in_seq=0 out_seq=0 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.813+0000 7f847b400640 10 client.25544 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f847b400640 1 -- 172.21.15.99:0/1899508461 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a07de44800 con 0x55a07dee8190 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f847b400640 10 client.25544 handle_command_reply: tid=0 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 client.25544 shutdown 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 msgr2=0x55a07dedcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8480005890 tx=0x7f848000f870 comp rx=0 tx=0).stop 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 shutdown_connections 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07dee8190 0x55a07deea580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07dee4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dedcff0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 --2- 172.21.15.99:0/1899508461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 >> 172.21.15.99:0/1899508461 conn(0x55a07decdc00 msgr2=0x55a07decceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f847b400640 0 client.25544 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 shutdown_connections 2024-03-23T23:44:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 1 -- 172.21.15.99:0/1899508461 wait complete. 2024-03-23T23:44:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.814+0000 7f849f16a740 20 client.25544 trim_cache size 0 max 16384 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 Processor -- start 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 -- start start 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07deda8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07decdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a6880 con 0x55a07ded2810 2024-03-23T23:44:23.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a8200 con 0x55a07ded39e0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f849f16a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a6ee0 con 0x55a07ded43a0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.828+0000 7f8499000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56262/0 (socket says 172.21.15.99:56262) 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 -- 172.21.15.99:0/3756528423 learned_addr learned my addr 172.21.15.99:0/3756528423 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f849a400640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07decdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499a00640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07deda8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 -- 172.21.15.99:0/3756528423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 msgr2=0x55a07decdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07decdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 -- 172.21.15.99:0/3756528423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 msgr2=0x55a07deda8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07deda8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 -- 172.21.15.99:0/3756528423 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07deda570 con 0x55a07ded39e0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f849a400640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07decdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8499000640 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ea0 tx=0x7f8480011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62df27f4c5f557d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8486000640 1 -- 172.21.15.99:0/3756528423 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480027070 con 0x55a07ded39e0 2024-03-23T23:44:23.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8486000640 1 -- 172.21.15.99:0/3756528423 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8480005520 con 0x55a07ded39e0 2024-03-23T23:44:23.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.829+0000 7f8486000640 1 -- 172.21.15.99:0/3756528423 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480031080 con 0x55a07ded39e0 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 -- 172.21.15.99:0/3756528423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 msgr2=0x55a07dee3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ea0 tx=0x7f8480011ab0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 -- 172.21.15.99:0/3756528423 shutdown_connections 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded43a0 0x55a07decdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded39e0 0x55a07dee3ff0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 --2- 172.21.15.99:0/3756528423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07deda8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 -- 172.21.15.99:0/3756528423 >> 172.21.15.99:0/3756528423 conn(0x55a07decdc00 msgr2=0x55a07ded16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 -- 172.21.15.99:0/3756528423 shutdown_connections 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.830+0000 7f849f16a740 1 -- 172.21.15.99:0/3756528423 wait complete. 2024-03-23T23:44:23.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 -- start start 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 Processor -- start 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded43a0 0x55a07dee2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a7ba0 con 0x55a07ded2810 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a7540 con 0x55a07ded43a0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941a4a40 con 0x55a07ded39e0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.831+0000 7f849f16a740 10 client.0 _ll_register_callbacks cb 0x7f848001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48814/0 (socket says 172.21.15.99:48814) 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 -- 172.21.15.99:0/132360661 learned_addr learned my addr 172.21.15.99:0/132360661 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499a00640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f849a400640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded43a0 0x55a07dee2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 -- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded43a0 msgr2=0x55a07dee2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded43a0 0x55a07dee2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 -- 172.21.15.99:0/132360661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 msgr2=0x55a07dedf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 -- 172.21.15.99:0/132360661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a07dee33d0 con 0x55a07ded39e0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f8499000640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8480004940 tx=0x7f8480019ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfcfab23d69e4a0f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.832+0000 7f847b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480039070 con 0x55a07ded39e0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8480005240 con 0x55a07ded39e0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8480005520 con 0x55a07ded39e0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8480027070 con 0x55a07ded39e0 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f849f16a740 10 client.25576 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:23.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a07df43ae0 con 0x55a07ded39e0 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.833+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8480005520 con 0x55a07ded39e0 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8480021040 con 0x55a07ded39e0 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 10 client.25576 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 20 client.25576 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07df44310 0x55a07df46730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:23.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 4 client.25576 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a07de3eb50 con 0x55a07df44310 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f8499a00640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07df44310 0x55a07df46730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f847b400640 10 client.25576 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.834+0000 7f8499a00640 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07df44310 0x55a07df46730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05168f2803f1f3c server_cookie=a14a7c04bd00d96f in_seq=0 out_seq=0 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f847b400640 1 -- 172.21.15.99:0/132360661 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a07de3eb50 con 0x55a07df44310 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f847b400640 10 client.25576 handle_command_reply: tid=0 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 client.25576 shutdown 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 msgr2=0x55a07dedfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8480004940 tx=0x7f8480019ea0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 shutdown_connections 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a07df44310 0x55a07df46730 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a07ded43a0 0x55a07dee2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f847b400640 0 client.25576 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a07ded39e0 0x55a07dedfdc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 --2- 172.21.15.99:0/132360661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a07ded2810 0x55a07dedf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 >> 172.21.15.99:0/132360661 conn(0x55a07decdc00 msgr2=0x55a07ded0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 shutdown_connections 2024-03-23T23:44:23.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 1 -- 172.21.15.99:0/132360661 wait complete. 2024-03-23T23:44:23.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.835+0000 7f849f16a740 20 client.25576 trim_cache size 0 max 16384 2024-03-23T23:44:23.838 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 msgr2=0x7f847c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 0x7f847c0a4d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55a07ded38e0 tx=0x7f848c009500 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 msgr2=0x7f8494199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f848802c650 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 shutdown_connections 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f847c0a2880 0x7f847c0a4d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8494103260 0x7f84941a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84941028a0 0x7f849419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 --2- 172.21.15.99:0/3368523602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84941016b0 0x7f8494199c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:23.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 >> 172.21.15.99:0/3368523602 conn(0x7f84940fcc40 msgr2=0x7f8494108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:23.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 shutdown_connections 2024-03-23T23:44:23.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:23.836+0000 7f849c400640 1 -- 172.21.15.99:0/3368523602 wait complete. 2024-03-23T23:44:24.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:23 smithi099 ceph-mon[31715]: pgmap v415: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-23T23:44:24.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:23 smithi149 ceph-mon[36083]: pgmap v415: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-23T23:44:24.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:23 smithi174 ceph-mon[37002]: pgmap v415: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-23T23:44:24.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T23:44:24.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:44:24.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T23:44:25.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 -- 172.21.15.99:0/1036416261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 msgr2=0x7f00301019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 --2- 172.21.15.99:0/1036416261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301019c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 -- 172.21.15.99:0/1036416261 shutdown_connections 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 --2- 172.21.15.99:0/1036416261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 0x7f00301099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 --2- 172.21.15.99:0/1036416261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 0x7f0030102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 --2- 172.21.15.99:0/1036416261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301019c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 -- 172.21.15.99:0/1036416261 >> 172.21.15.99:0/1036416261 conn(0x7f00300fcb10 msgr2=0x7f00300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 -- 172.21.15.99:0/1036416261 shutdown_connections 2024-03-23T23:44:25.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.082+0000 7f0036400640 1 -- 172.21.15.99:0/1036416261 wait complete. 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.083+0000 7f0036400640 1 Processor -- start 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 -- start start 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 0x7f0030197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 0x7f003019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003010d240 con 0x7f0030102790 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003010d0c0 con 0x7f00301015a0 2024-03-23T23:44:25.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0036400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003010d3c0 con 0x7f0030103150 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0035000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0035000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56288/0 (socket says 172.21.15.99:56288) 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0035000640 1 -- 172.21.15.99:0/520984140 learned_addr learned my addr 172.21.15.99:0/520984140 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0035a00640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 0x7f003019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.084+0000 7f0027e00640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 0x7f0030197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 -- 172.21.15.99:0/520984140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 msgr2=0x7f003019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 0x7f003019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 -- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 msgr2=0x7f0030197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 0x7f0030197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 -- 172.21.15.99:0/520984140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003019f4d0 con 0x7f00301015a0 2024-03-23T23:44:25.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0035000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f002002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f002e000640 1 -- 172.21.15.99:0/520984140 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020043020 con 0x7f00301015a0 2024-03-23T23:44:25.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f002e000640 1 -- 172.21.15.99:0/520984140 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00200054e0 con 0x7f00301015a0 2024-03-23T23:44:25.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f002e000640 1 -- 172.21.15.99:0/520984140 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020003ce0 con 0x7f00301015a0 2024-03-23T23:44:25.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.085+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003019f760 con 0x7f00301015a0 2024-03-23T23:44:25.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.086+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003019fc30 con 0x7f00301015a0 2024-03-23T23:44:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 Processor -- start 2024-03-23T23:44:25.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 -- start start 2024-03-23T23:44:25.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342b0e10 0x5647342b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734210eb0 con 0x5647342abca0 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f002e000640 1 -- 172.21.15.99:0/520984140 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0020046020 con 0x7f00301015a0 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647341b9a60 con 0x5647342b0e10 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.090+0000 7f003a7a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56473420cf50 con 0x5647342ab2b0 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48840/0 (socket says 172.21.15.99:48840) 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 -- 172.21.15.99:0/3911928733 learned_addr learned my addr 172.21.15.99:0/3911928733 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0027e00640 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342b0e10 0x5647342b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035000640 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 -- 172.21.15.99:0/3911928733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342b0e10 msgr2=0x5647342b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342b0e10 0x5647342b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 -- 172.21.15.99:0/3911928733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 msgr2=0x5647342b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 -- 172.21.15.99:0/3911928733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564733fed7e0 con 0x5647342ab2b0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.091+0000 7f0035a00640 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0028009fd0 tx=0x7f00280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20ebde98b71706a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f0026000640 1 -- 172.21.15.99:0/3911928733 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028004d20 con 0x5647342ab2b0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f0026000640 1 -- 172.21.15.99:0/3911928733 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f002801a070 con 0x5647342ab2b0 2024-03-23T23:44:25.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f0026000640 1 -- 172.21.15.99:0/3911928733 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028016570 con 0x5647342ab2b0 2024-03-23T23:44:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f002e000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 0x7f00140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 -- 172.21.15.99:0/3911928733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 msgr2=0x5647342ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0028009fd0 tx=0x7f00280118d0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f002e000640 1 -- 172.21.15.99:0/520984140 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f00200f0b50 con 0x7f00301015a0 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 -- 172.21.15.99:0/3911928733 shutdown_connections 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342b0e10 0x5647342b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3911928733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342ab2b0 0x5647342ab6d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.092+0000 7f003a7a4740 1 -- 172.21.15.99:0/3911928733 >> 172.21.15.99:0/3911928733 conn(0x5647342a6820 msgr2=0x5647342a6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f0035000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 0x7f00140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f003a7a4740 1 -- 172.21.15.99:0/3911928733 shutdown_connections 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f003a7a4740 1 -- 172.21.15.99:0/3911928733 wait complete. 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f003a7a4740 1 -- start start 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f0035000640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 0x7f00140a4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f00200b98d0 tx=0x7f00200b9040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.093+0000 7f003a7a4740 1 Processor -- start 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647342b3dc0 con 0x5647342ab2b0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647341cceb0 con 0x5647342abca0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647342b3f40 con 0x5647342b0e10 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f003a7a4740 10 client.0 _ll_register_callbacks cb 0x5647341c1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f0027e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f0027e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56314/0 (socket says 172.21.15.99:56314) 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f0027e00640 1 -- 172.21.15.99:0/887417256 learned_addr learned my addr 172.21.15.99:0/887417256 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f0035a00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.094+0000 7f0035a00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 -- 172.21.15.99:0/887417256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 msgr2=0x5647342bca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 -- 172.21.15.99:0/887417256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 msgr2=0x5647342b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 -- 172.21.15.99:0/887417256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56473404df70 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0035a00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f0027e00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f001800bad0 tx=0x7f001800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db0dcf7b6343ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f001ea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018003e20 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0018011440 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.095+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f002801a070 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0018011640 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 10 client.25562 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564733ee5170 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f002801a070 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0018025ad0 con 0x5647342abca0 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 10 client.25562 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:25.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 20 client.25562 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5647342c1210 0x5647342c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 4 client.25562 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.096+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56473421e3f0 con 0x5647342c1210 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.097+0000 7f0035a00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5647342c1210 0x5647342c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.097+0000 7f0035a00640 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5647342c1210 0x5647342c3670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1976b8b31f7680 server_cookie=176ba885dc5ba269 in_seq=0 out_seq=0 2024-03-23T23:44:25.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.097+0000 7f001ea00640 10 client.25562 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f001ea00640 1 -- 172.21.15.99:0/887417256 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56473421e3f0 con 0x5647342c1210 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f001ea00640 10 client.25562 handle_command_reply: tid=0 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 client.25562 shutdown 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 msgr2=0x5647342b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f001800bad0 tx=0x7f001800bf90 comp rx=0 tx=0).stop 2024-03-23T23:44:25.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 shutdown_connections 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5647342c1210 0x5647342c3670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342b5a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f001ea00640 0 client.25562 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 --2- 172.21.15.99:0/887417256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 >> 172.21.15.99:0/887417256 conn(0x5647342a6820 msgr2=0x5647342a5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 shutdown_connections 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.098+0000 7f003a7a4740 1 -- 172.21.15.99:0/887417256 wait complete. 2024-03-23T23:44:25.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.099+0000 7f003a7a4740 20 client.25562 trim_cache size 0 max 16384 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.124+0000 7f003a7a4740 1 Processor -- start 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.124+0000 7f003a7a4740 1 -- start start 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a6650 con 0x5647342ab2b0 2024-03-23T23:44:25.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a7fd0 con 0x5647342abca0 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f003a7a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a6cb0 con 0x5647342b0e10 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f0027e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f0035a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f0027e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56318/0 (socket says 172.21.15.99:56318) 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f0027e00640 1 -- 172.21.15.99:0/1757593164 learned_addr learned my addr 172.21.15.99:0/1757593164 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.125+0000 7f0035a00640 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 -- 172.21.15.99:0/1757593164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 msgr2=0x5647342c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 -- 172.21.15.99:0/1757593164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 msgr2=0x5647342aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 -- 172.21.15.99:0/1757593164 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647342a7290 con 0x5647342abca0 2024-03-23T23:44:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0027e00640 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0018025ec0 tx=0x7f00180115d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf59257a6df55660 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0025600640 1 -- 172.21.15.99:0/1757593164 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180117e0 con 0x5647342abca0 2024-03-23T23:44:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0025600640 1 -- 172.21.15.99:0/1757593164 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0018005040 con 0x5647342abca0 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.126+0000 7f0025600640 1 -- 172.21.15.99:0/1757593164 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018025800 con 0x5647342abca0 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 -- 172.21.15.99:0/1757593164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 msgr2=0x5647342bca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0018025ec0 tx=0x7f00180115d0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 -- 172.21.15.99:0/1757593164 shutdown_connections 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342abca0 0x5647342bca00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 --2- 172.21.15.99:0/1757593164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342ab2b0 0x5647342aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 -- 172.21.15.99:0/1757593164 >> 172.21.15.99:0/1757593164 conn(0x5647342a6820 msgr2=0x5647342a5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 -- 172.21.15.99:0/1757593164 shutdown_connections 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.127+0000 7f003a7a4740 1 -- 172.21.15.99:0/1757593164 wait complete. 2024-03-23T23:44:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 -- start start 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 Processor -- start 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a7970 con 0x5647342abca0 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a7310 con 0x5647342ab2b0 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00301a4810 con 0x5647342b0e10 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.128+0000 7f003a7a4740 10 client.0 _ll_register_callbacks cb 0x7f0018004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56332/0 (socket says 172.21.15.99:56332) 2024-03-23T23:44:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 -- 172.21.15.99:0/3034951022 learned_addr learned my addr 172.21.15.99:0/3034951022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0027e00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 -- 172.21.15.99:0/3034951022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 msgr2=0x5647342bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 -- 172.21.15.99:0/3034951022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 msgr2=0x5647342bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 -- 172.21.15.99:0/3034951022 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564734325310 con 0x5647342ab2b0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0027e00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f0035a00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0028009de0 tx=0x7f0028009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4275894cdb3d2061 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f001ea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.129+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028004420 con 0x5647342ab2b0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f002802e110 con 0x5647342ab2b0 2024-03-23T23:44:25.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0018005040 con 0x5647342ab2b0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00280045c0 con 0x5647342ab2b0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 10 client.25568 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564734326530 con 0x5647342ab2b0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0018005040 con 0x5647342ab2b0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f0028029040 con 0x5647342ab2b0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 10 client.25568 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 20 client.25568 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564734326bb0 0x564734328fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 4 client.25568 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.130+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564734218740 con 0x564734326bb0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.131+0000 7f0027e00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564734326bb0 0x564734328fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.131+0000 7f0027e00640 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564734326bb0 0x564734328fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c4b62b14186740 server_cookie=625c841bc659ef96 in_seq=0 out_seq=0 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.131+0000 7f001ea00640 10 client.25568 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f001ea00640 1 -- 172.21.15.99:0/3034951022 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564734218740 con 0x564734326bb0 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f001ea00640 10 client.25568 handle_command_reply: tid=0 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 client.25568 shutdown 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 msgr2=0x5647342bb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0028009de0 tx=0x7f0028009230 comp rx=0 tx=0).stop 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 shutdown_connections 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564734326bb0 0x564734328fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647342b0e10 0x5647342bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647342abca0 0x5647342bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 --2- 172.21.15.99:0/3034951022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647342ab2b0 0x5647342bb0f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 >> 172.21.15.99:0/3034951022 conn(0x5647342a6820 msgr2=0x5647342ac740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.132+0000 7f001ea00640 0 client.25568 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 shutdown_connections 2024-03-23T23:44:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f003a7a4740 1 -- 172.21.15.99:0/3034951022 wait complete. 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f003a7a4740 20 client.25568 trim_cache size 0 max 16384 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "323427c4-9397-41b4-ab95-df6a2798dca1", 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-23T23:44:25.138 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 msgr2=0x7f00140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 0x7f00140a4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f00200b98d0 tx=0x7f00200b9040 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 msgr2=0x7f00301978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.133+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f002002c6c0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 shutdown_connections 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f00140a2880 0x7f00140a4d40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0030103150 0x7f003019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0030102790 0x7f0030197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 --2- 172.21.15.99:0/520984140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00301015a0 0x7f00301978b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 >> 172.21.15.99:0/520984140 conn(0x7f00300fcb10 msgr2=0x7f0030105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 shutdown_connections 2024-03-23T23:44:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.134+0000 7f0036400640 1 -- 172.21.15.99:0/520984140 wait complete. 2024-03-23T23:44:25.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:44:25.158 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 -- 172.21.15.99:0/4285038874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8401038e0 msgr2=0x7fd840103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 --2- 172.21.15.99:0/4285038874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8401038e0 0x7fd840103d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd830009900 tx=0x7fd83002bc40 comp rx=0 tx=0).stop 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 -- 172.21.15.99:0/4285038874 shutdown_connections 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 --2- 172.21.15.99:0/4285038874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840105490 0x7fd84010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 --2- 172.21.15.99:0/4285038874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 0x7fd840104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 --2- 172.21.15.99:0/4285038874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8401038e0 0x7fd840103d00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.347+0000 7fd845800640 1 -- 172.21.15.99:0/4285038874 >> 172.21.15.99:0/4285038874 conn(0x7fd8400fee50 msgr2=0x7fd840101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.348+0000 7fd845800640 1 -- 172.21.15.99:0/4285038874 shutdown_connections 2024-03-23T23:44:25.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.348+0000 7fd845800640 1 -- 172.21.15.99:0/4285038874 wait complete. 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 Processor -- start 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 -- start start 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 0x7fd84019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd840105490 0x7fd8401a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84010f5b0 con 0x7fd840104ad0 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84010f430 con 0x7fd840105490 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd845800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84010f730 con 0x7fd8401038e0 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48896/0 (socket says 172.21.15.99:48896) 2024-03-23T23:44:25.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd83ea00640 1 -- 172.21.15.99:0/4117428895 learned_addr learned my addr 172.21.15.99:0/4117428895 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd83f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd840105490 0x7fd8401a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.349+0000 7fd83e000640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 0x7fd84019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 -- 172.21.15.99:0/4117428895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd840105490 msgr2=0x7fd8401a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd840105490 0x7fd8401a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 -- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 msgr2=0x7fd84019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 0x7fd84019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 -- 172.21.15.99:0/4117428895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8401a1810 con 0x7fd8401038e0 2024-03-23T23:44:25.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd83ea00640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd83002efd0 tx=0x7fd83002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd837400640 1 -- 172.21.15.99:0/4117428895 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830041070 con 0x7fd8401038e0 2024-03-23T23:44:25.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd837400640 1 -- 172.21.15.99:0/4117428895 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd830004450 con 0x7fd8401038e0 2024-03-23T23:44:25.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.350+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8401a1aa0 con 0x7fd8401038e0 2024-03-23T23:44:25.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.351+0000 7fd837400640 1 -- 172.21.15.99:0/4117428895 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830003e50 con 0x7fd8401038e0 2024-03-23T23:44:25.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.351+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8401a1fb0 con 0x7fd8401038e0 2024-03-23T23:44:25.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 Processor -- start 2024-03-23T23:44:25.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd837400640 1 -- 172.21.15.99:0/4117428895 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd830044070 con 0x7fd8401038e0 2024-03-23T23:44:25.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 -- start start 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc0522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc052870 0x55bbcc057370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc059e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcbfb7250 con 0x55bbcc052870 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcbf5fa30 con 0x55bbcc051e80 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd84840c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcbfb32f0 con 0x55bbcc0579e0 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd83f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc0522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd83f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc0522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56370/0 (socket says 172.21.15.99:56370) 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd83f400640 1 -- 172.21.15.99:0/3066832170 learned_addr learned my addr 172.21.15.99:0/3066832170 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd83e000640 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc059e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.354+0000 7fd83ea00640 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc052870 0x55bbcc057370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 -- 172.21.15.99:0/3066832170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 msgr2=0x55bbcc0522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc0522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 -- 172.21.15.99:0/3066832170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc052870 msgr2=0x55bbcc057370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc052870 0x55bbcc057370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 -- 172.21.15.99:0/3066832170 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbcbf4a690 con 0x55bbcc0579e0 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd83e000640 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc059e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009bf0 tx=0x7fd82c011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3985ab0887298842 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd837400640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 0x7fd8240a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd837400640 1 -- 172.21.15.99:0/4117428895 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd8300f0590 con 0x7fd8401038e0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.355+0000 7fd82be00640 1 -- 172.21.15.99:0/3066832170 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c004310 con 0x55bbcc0579e0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd83f400640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 0x7fd8240a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd82be00640 1 -- 172.21.15.99:0/3066832170 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c0044b0 con 0x55bbcc0579e0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd82be00640 1 -- 172.21.15.99:0/3066832170 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c003850 con 0x55bbcc0579e0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd84840c740 1 -- 172.21.15.99:0/3066832170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 msgr2=0x55bbcc059e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd84840c740 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc059e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009bf0 tx=0x7fd82c011440 comp rx=0 tx=0).stop 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.356+0000 7fd83f400640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 0x7fd8240a4ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55bbcc0530d0 tx=0x7fd838009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 -- 172.21.15.99:0/3066832170 shutdown_connections 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc059e70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc052870 0x55bbcc057370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 --2- 172.21.15.99:0/3066832170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc0522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 -- 172.21.15.99:0/3066832170 >> 172.21.15.99:0/3066832170 conn(0x55bbcc01a200 msgr2=0x55bbcc04d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 -- 172.21.15.99:0/3066832170 shutdown_connections 2024-03-23T23:44:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 -- 172.21.15.99:0/3066832170 wait complete. 2024-03-23T23:44:25.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.357+0000 7fd84840c740 1 -- start start 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 Processor -- start 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc05c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc063380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcc05a990 con 0x55bbcc0579e0 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcbf72e80 con 0x55bbcc051e80 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbcc05ab10 con 0x55bbcc052870 2024-03-23T23:44:25.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd84840c740 10 client.0 _ll_register_callbacks cb 0x55bbcbf85130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd83e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc05c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd83f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:56380/0 (socket says 172.21.15.99:56380) 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.358+0000 7fd83ea00640 1 -- 172.21.15.99:0/2952553147 learned_addr learned my addr 172.21.15.99:0/2952553147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 -- 172.21.15.99:0/2952553147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 msgr2=0x55bbcc05c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc05c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 -- 172.21.15.99:0/2952553147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 msgr2=0x55bbcc063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc063380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 -- 172.21.15.99:0/2952553147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbcbd957d0 con 0x55bbcc051e80 2024-03-23T23:44:25.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd83ea00640 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd83002c180 tx=0x7fd830005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb30162657740d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd828c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83004b070 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd8300c5220 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd82c0044b0 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.359+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8300bcb00 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 10 client.25580 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bbcbccb3f0 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c0044b0 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd8300d2050 con 0x55bbcc051e80 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 10 client.25580 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:25.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 20 client.25580 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc067c30 0x55bbcc06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 4 client.25580 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbcbfc4790 con 0x55bbcc067c30 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.360+0000 7fd83e000640 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc067c30 0x55bbcc06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.361+0000 7fd83e000640 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc067c30 0x55bbcc06a0d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce508668d838f41 server_cookie=c4f4ad96a3918b85 in_seq=0 out_seq=0 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.361+0000 7fd828c00640 10 client.25580 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd828c00640 1 -- 172.21.15.99:0/2952553147 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bbcbfc4790 con 0x55bbcc067c30 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd828c00640 10 client.25580 handle_command_reply: tid=0 2024-03-23T23:44:25.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 client.25580 shutdown 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 msgr2=0x55bbcc05be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd83002c180 tx=0x7fd830005c90 comp rx=0 tx=0).stop 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 shutdown_connections 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc067c30 0x55bbcc06a0d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc063380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc05c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd828c00640 0 client.25580 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 --2- 172.21.15.99:0/2952553147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc05be00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 >> 172.21.15.99:0/2952553147 conn(0x55bbcc01a200 msgr2=0x55bbcc04ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 shutdown_connections 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 1 -- 172.21.15.99:0/2952553147 wait complete. 2024-03-23T23:44:25.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.362+0000 7fd84840c740 20 client.25580 trim_cache size 0 max 16384 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 Processor -- start 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 -- start start 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 0x55bbcc05c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 0x55bbcc063310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc0531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401aac50 con 0x55bbcc051e80 2024-03-23T23:44:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401ac5d0 con 0x55bbcc052870 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd84840c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401ab2b0 con 0x55bbcc0579e0 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd83f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc0531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 0x55bbcc05c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd83e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 0x55bbcc063310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd83f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc0531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48922/0 (socket says 172.21.15.99:48922) 2024-03-23T23:44:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.388+0000 7fd83f400640 1 -- 172.21.15.99:0/2917184295 learned_addr learned my addr 172.21.15.99:0/2917184295 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 -- 172.21.15.99:0/2917184295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 msgr2=0x55bbcc0531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc0531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 -- 172.21.15.99:0/2917184295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 msgr2=0x55bbcc05c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 0x55bbcc05c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 -- 172.21.15.99:0/2917184295 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbcbc8b0f0 con 0x55bbcc052870 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83ea00640 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 0x55bbcc05c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd83e000640 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 0x55bbcc063310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd82c011940 tx=0x7fd82c004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f2c4ee5cf55ed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd82b400640 1 -- 172.21.15.99:0/2917184295 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c030070 con 0x55bbcc052870 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.389+0000 7fd82b400640 1 -- 172.21.15.99:0/2917184295 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c029030 con 0x55bbcc052870 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd82b400640 1 -- 172.21.15.99:0/2917184295 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0035f0 con 0x55bbcc052870 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 -- 172.21.15.99:0/2917184295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 msgr2=0x55bbcc063310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 0x55bbcc063310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd82c011940 tx=0x7fd82c004230 comp rx=0 tx=0).stop 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 -- 172.21.15.99:0/2917184295 shutdown_connections 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc0579e0 0x55bbcc0531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc052870 0x55bbcc063310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 --2- 172.21.15.99:0/2917184295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc051e80 0x55bbcc05c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 -- 172.21.15.99:0/2917184295 >> 172.21.15.99:0/2917184295 conn(0x55bbcc01a200 msgr2=0x55bbcc05ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 -- 172.21.15.99:0/2917184295 shutdown_connections 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.390+0000 7fd84840c740 1 -- 172.21.15.99:0/2917184295 wait complete. 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 -- start start 2024-03-23T23:44:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 Processor -- start 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc060960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc061eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401abf70 con 0x55bbcc0579e0 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401ab910 con 0x55bbcc051e80 2024-03-23T23:44:25.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401a8e10 con 0x55bbcc052870 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.391+0000 7fd84840c740 10 client.0 _ll_register_callbacks cb 0x7fd830005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc060960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48932/0 (socket says 172.21.15.99:48932) 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 -- 172.21.15.99:0/3419081581 learned_addr learned my addr 172.21.15.99:0/3419081581 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83f400640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc061eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 -- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 msgr2=0x55bbcc060960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc060960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 -- 172.21.15.99:0/3419081581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 msgr2=0x55bbcc061eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc061eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 -- 172.21.15.99:0/3419081581 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbcc0624a0 con 0x55bbcc052870 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83ea00640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc060960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd83e000640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0042e0 tx=0x7fd82c0119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbf20893ce73a64 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd828c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:25.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.392+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c030070 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd82c029030 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd82c02a280 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82c005620 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 10 client.25612 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bbcc0c3560 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c029030 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd82c018070 con 0x55bbcc052870 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 10 client.25612 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 20 client.25612 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc0c3b90 0x55bbcc0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 4 client.25612 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.393+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbcbfbeae0 con 0x55bbcc0c3b90 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd83ea00640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc0c3b90 0x55bbcc0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd83ea00640 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc0c3b90 0x55bbcc0c5fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25cb35c769434c3c server_cookie=170644cb5d045e98 in_seq=0 out_seq=0 2024-03-23T23:44:25.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd828c00640 10 client.25612 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd828c00640 1 -- 172.21.15.99:0/3419081581 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bbcbfbeae0 con 0x55bbcc0c3b90 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd828c00640 10 client.25612 handle_command_reply: tid=0 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd84840c740 1 client.25612 shutdown 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 msgr2=0x55bbcc060ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0042e0 tx=0x7fd82c0119f0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 shutdown_connections 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.394+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bbcc0c3b90 0x55bbcc0c5fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbcc0579e0 0x55bbcc061eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbcc052870 0x55bbcc060ea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd828c00640 0 client.25612 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 --2- 172.21.15.99:0/3419081581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbcc051e80 0x55bbcc060960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 >> 172.21.15.99:0/3419081581 conn(0x55bbcc01a200 msgr2=0x55bbcc051140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 shutdown_connections 2024-03-23T23:44:25.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 1 -- 172.21.15.99:0/3419081581 wait complete. 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd84840c740 20 client.25612 trim_cache size 0 max 16384 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stdout: "323427c4-9397-41b4-ab95-df6a2798dca1": { 2024-03-23T23:44:25.399 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-23T23:44:25.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "323427c4-9397-41b4-ab95-df6a2798dca1", 2024-03-23T23:44:25.400 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-23T23:44:25.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:44:25.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:44:25.400 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 msgr2=0x7fd8240a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 0x7fd8240a4ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55bbcc0530d0 tx=0x7fd838009500 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 msgr2=0x7fd840199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.395+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd83002efd0 tx=0x7fd83002c660 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 shutdown_connections 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd8240a2a20 0x7fd8240a4ee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd840105490 0x7fd8401a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd840104ad0 0x7fd84019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 --2- 172.21.15.99:0/4117428895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8401038e0 0x7fd840199bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 >> 172.21.15.99:0/4117428895 conn(0x7fd8400fee50 msgr2=0x7fd840107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 shutdown_connections 2024-03-23T23:44:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:25.396+0000 7fd845800640 1 -- 172.21.15.99:0/4117428895 wait complete. 2024-03-23T23:44:25.412 INFO:tasks.cephfs.filesystem:scrub status for tag:323427c4-9397-41b4-ab95-df6a2798dca1 - {'path': '/', 'tag': '323427c4-9397-41b4-ab95-df6a2798dca1', 'options': 'recursive,force'} 2024-03-23T23:44:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:25 smithi149 ceph-mon[36083]: pgmap v416: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-23T23:44:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:25 smithi149 ceph-mon[36083]: scrub queued for path: / 2024-03-23T23:44:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:25 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:25 smithi149 ceph-mon[36083]: scrub summary: active paths [/] 2024-03-23T23:44:26.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:25 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:25 smithi174 ceph-mon[37002]: pgmap v416: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-23T23:44:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:25 smithi174 ceph-mon[37002]: scrub queued for path: / 2024-03-23T23:44:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:25 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:25 smithi174 ceph-mon[37002]: scrub summary: active paths [/] 2024-03-23T23:44:26.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:25 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:25 smithi099 ceph-mon[31715]: pgmap v416: 97 pgs: 97 active+clean; 662 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-03-23T23:44:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:25 smithi099 ceph-mon[31715]: scrub queued for path: / 2024-03-23T23:44:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:25 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:26.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:25 smithi099 ceph-mon[31715]: scrub summary: active paths [/] 2024-03-23T23:44:26.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:25 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:44:28.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:27 smithi099 ceph-mon[31715]: pgmap v417: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:27 smithi149 ceph-mon[36083]: pgmap v417: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:28.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:27 smithi174 ceph-mon[37002]: pgmap v417: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:29.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:28 smithi149 ceph-mon[36083]: scrub summary: idle 2024-03-23T23:44:29.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:28 smithi174 ceph-mon[37002]: scrub summary: idle 2024-03-23T23:44:29.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:28 smithi099 ceph-mon[31715]: scrub summary: idle 2024-03-23T23:44:30.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:29 smithi149 ceph-mon[36083]: pgmap v418: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:30.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:29 smithi174 ceph-mon[37002]: pgmap v418: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:29 smithi099 ceph-mon[31715]: pgmap v418: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-23T23:44:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 -- 172.21.15.99:0/3855914946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 msgr2=0x7fea88100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 --2- 172.21.15.99:0/3855914946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88100180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 -- 172.21.15.99:0/3855914946 shutdown_connections 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 --2- 172.21.15.99:0/3855914946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea88101060 0x7fea881014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 --2- 172.21.15.99:0/3855914946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea881006c0 0x7fea88100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 --2- 172.21.15.99:0/3855914946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88100180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 -- 172.21.15.99:0/3855914946 >> 172.21.15.99:0/3855914946 conn(0x7fea880fb980 msgr2=0x7fea880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 -- 172.21.15.99:0/3855914946 shutdown_connections 2024-03-23T23:44:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.817+0000 7fea8d800640 1 -- 172.21.15.99:0/3855914946 wait complete. 2024-03-23T23:44:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 Processor -- start 2024-03-23T23:44:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 -- start start 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea881006c0 0x7fea88199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea88101060 0x7fea881a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810f380 con 0x7fea881006c0 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810f200 con 0x7fea88101060 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.819+0000 7fea8d800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810f500 con 0x7fea880ffd60 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea881006c0 0x7fea88199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56218/0 (socket says 172.21.15.99:56218) 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 -- 172.21.15.99:0/228820523 learned_addr learned my addr 172.21.15.99:0/228820523 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea8ce00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea88101060 0x7fea881a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 -- 172.21.15.99:0/228820523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea88101060 msgr2=0x7fea881a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea88101060 0x7fea881a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 -- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea881006c0 msgr2=0x7fea88199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea881006c0 0x7fea88199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 -- 172.21.15.99:0/228820523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea881a15b0 con 0x7fea880ffd60 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.820+0000 7fea87e00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea78000c00 tx=0x7fea7802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.821+0000 7fea84c00640 1 -- 172.21.15.99:0/228820523 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78043020 con 0x7fea880ffd60 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.821+0000 7fea84c00640 1 -- 172.21.15.99:0/228820523 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea780054e0 con 0x7fea880ffd60 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.821+0000 7fea84c00640 1 -- 172.21.15.99:0/228820523 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78003c40 con 0x7fea880ffd60 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.821+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea881a1840 con 0x7fea880ffd60 2024-03-23T23:44:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.821+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881a1d40 con 0x7fea880ffd60 2024-03-23T23:44:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.825+0000 7fea91a1b740 1 Processor -- start 2024-03-23T23:44:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.825+0000 7fea91a1b740 1 -- start start 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf8528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf8553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf8103b0 con 0x559bdf84dde0 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf7b2830 con 0x559bdf84d3f0 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea91a1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf75b5d0 con 0x559bdf852f50 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55510/0 (socket says 172.21.15.99:55510) 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea8ce00640 1 -- 172.21.15.99:0/3575074043 learned_addr learned my addr 172.21.15.99:0/3575074043 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea87400640 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf8553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.826+0000 7fea84c00640 1 -- 172.21.15.99:0/228820523 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fea78046020 con 0x7fea880ffd60 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 -- 172.21.15.99:0/3575074043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 msgr2=0x559bdf8553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf8553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 -- 172.21.15.99:0/3575074043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 msgr2=0x559bdf8528e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf8528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 -- 172.21.15.99:0/3575074043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bdf7732c0 con 0x559bdf84d3f0 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea8ce00640 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17fc1a4970539000 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea7d600640 1 -- 172.21.15.99:0/3575074043 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80004d20 con 0x559bdf84d3f0 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea7d600640 1 -- 172.21.15.99:0/3575074043 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea80004050 con 0x559bdf84d3f0 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.827+0000 7fea7d600640 1 -- 172.21.15.99:0/3575074043 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80005280 con 0x559bdf84d3f0 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 -- 172.21.15.99:0/3575074043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 msgr2=0x559bdf84d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea800118d0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 -- 172.21.15.99:0/3575074043 shutdown_connections 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf8553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf8528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 --2- 172.21.15.99:0/3575074043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf84d810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 -- 172.21.15.99:0/3575074043 >> 172.21.15.99:0/3575074043 conn(0x559bdf848960 msgr2=0x559bdf848d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 -- 172.21.15.99:0/3575074043 shutdown_connections 2024-03-23T23:44:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.828+0000 7fea91a1b740 1 -- 172.21.15.99:0/3575074043 wait complete. 2024-03-23T23:44:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.829+0000 7fea84c00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 0x7fea6c0a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.829+0000 7fea91a1b740 1 -- start start 2024-03-23T23:44:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.829+0000 7fea84c00640 1 -- 172.21.15.99:0/228820523 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fea780f04b0 con 0x7fea880ffd60 2024-03-23T23:44:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.829+0000 7fea91a1b740 1 Processor -- start 2024-03-23T23:44:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea8ce00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 0x7fea6c0a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf857b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf85eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf76ea20 con 0x559bdf84dde0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf7ae8d0 con 0x559bdf84d3f0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bdf855f00 con 0x559bdf852f50 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.830+0000 7fea91a1b740 10 client.0 _ll_register_callbacks cb 0x559bdf7ab1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55524/0 (socket says 172.21.15.99:55524) 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 -- 172.21.15.99:0/1810066339 learned_addr learned my addr 172.21.15.99:0/1810066339 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea8ce00640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf85eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87400640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf857b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 -- 172.21.15.99:0/1810066339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 msgr2=0x559bdf85eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea8ce00640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 0x7fea6c0a4c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea8001b040 tx=0x7fea80000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf85eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 -- 172.21.15.99:0/1810066339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 msgr2=0x559bdf857b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.831+0000 7fea87e00640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf857b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea87e00640 1 -- 172.21.15.99:0/1810066339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bdf4c7210 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea8ce00640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf85eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea87400640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf857b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea87e00640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea780b9230 tx=0x7fea780bc7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c6071058dc84b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea72000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78005690 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fea780ee040 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea80004050 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea78004c60 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea91a1b740 10 client.25598 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.832+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559bdf593620 con 0x559bdf84d3f0 2024-03-23T23:44:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea80004050 con 0x559bdf84d3f0 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fea780c50e0 con 0x559bdf84d3f0 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 10 client.25598 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 20 client.25598 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf863390 0x559bdf865780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 4 client.25598 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bdf7bfd70 con 0x559bdf863390 2024-03-23T23:44:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.833+0000 7fea87400640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf863390 0x559bdf865780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.834+0000 7fea72000640 10 client.25598 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.834+0000 7fea87400640 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf863390 0x559bdf865780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f5eae4de4c8c70 server_cookie=b9c31e8d8b0c0923 in_seq=0 out_seq=0 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea72000640 1 -- 172.21.15.99:0/1810066339 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559bdf7bfd70 con 0x559bdf863390 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea72000640 10 client.25598 handle_command_reply: tid=0 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 client.25598 shutdown 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 msgr2=0x559bdf8575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea780b9230 tx=0x7fea780bc7f0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 shutdown_connections 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf863390 0x559bdf865780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf852f50 0x559bdf85eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf84dde0 0x559bdf857b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 --2- 172.21.15.99:0/1810066339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8575e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea72000640 0 client.25598 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 >> 172.21.15.99:0/1810066339 conn(0x559bdf848960 msgr2=0x559bdf847b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 shutdown_connections 2024-03-23T23:44:30.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.835+0000 7fea91a1b740 1 -- 172.21.15.99:0/1810066339 wait complete. 2024-03-23T23:44:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.836+0000 7fea91a1b740 20 client.25598 trim_cache size 0 max 16384 2024-03-23T23:44:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.854+0000 7fea91a1b740 1 Processor -- start 2024-03-23T23:44:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 -- start start 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf84bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881aaa00 con 0x559bdf852f50 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881ac380 con 0x559bdf84d3f0 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea91a1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881ab060 con 0x559bdf84dde0 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56256/0 (socket says 172.21.15.99:56256) 2024-03-23T23:44:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 -- 172.21.15.99:0/4263274707 learned_addr learned my addr 172.21.15.99:0/4263274707 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea8ce00640 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf84bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 -- 172.21.15.99:0/4263274707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 msgr2=0x559bdf8436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 -- 172.21.15.99:0/4263274707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 msgr2=0x559bdf84bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.855+0000 7fea87400640 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf84bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.856+0000 7fea87400640 1 -- 172.21.15.99:0/4263274707 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bdf865af0 con 0x559bdf84dde0 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.856+0000 7fea87400640 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x559bdf8498d0 tx=0x7fea74005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e8a2375644114d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.856+0000 7fea7cc00640 1 -- 172.21.15.99:0/4263274707 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74020280 con 0x559bdf84dde0 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.856+0000 7fea7cc00640 1 -- 172.21.15.99:0/4263274707 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea74020e40 con 0x559bdf84dde0 2024-03-23T23:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.856+0000 7fea7cc00640 1 -- 172.21.15.99:0/4263274707 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74003970 con 0x559bdf84dde0 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 -- 172.21.15.99:0/4263274707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 msgr2=0x559bdf85eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x559bdf8498d0 tx=0x7fea74005f90 comp rx=0 tx=0).stop 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 -- 172.21.15.99:0/4263274707 shutdown_connections 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf84bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85eaf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4263274707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf8436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 -- 172.21.15.99:0/4263274707 >> 172.21.15.99:0/4263274707 conn(0x559bdf848960 msgr2=0x559bdf848d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.857+0000 7fea91a1b740 1 -- 172.21.15.99:0/4263274707 shutdown_connections 2024-03-23T23:44:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.858+0000 7fea91a1b740 1 -- 172.21.15.99:0/4263274707 wait complete. 2024-03-23T23:44:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.858+0000 7fea91a1b740 1 -- start start 2024-03-23T23:44:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.858+0000 7fea91a1b740 1 Processor -- start 2024-03-23T23:44:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf85d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881abd20 con 0x559bdf852f50 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881ab6c0 con 0x559bdf84d3f0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a8bc0 con 0x559bdf84dde0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea91a1b740 10 client.0 _ll_register_callbacks cb 0x7fea780bcc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55546/0 (socket says 172.21.15.99:55546) 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea87e00640 1 -- 172.21.15.99:0/4265024106 learned_addr learned my addr 172.21.15.99:0/4265024106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea8ce00640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf85d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.859+0000 7fea87400640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 -- 172.21.15.99:0/4265024106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 msgr2=0x559bdf85c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 -- 172.21.15.99:0/4265024106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 msgr2=0x559bdf85d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf85d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 -- 172.21.15.99:0/4265024106 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bdf85ddd0 con 0x559bdf84d3f0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87400640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea87e00640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea780095c0 tx=0x7fea780092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c4dd74463cf91 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea72000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7809f030 con 0x559bdf84d3f0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.860+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fea780edfb0 con 0x559bdf84d3f0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bdf7ba0c0 con 0x559bdf84d3f0 2024-03-23T23:44:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea78005690 con 0x559bdf84d3f0 2024-03-23T23:44:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 10 client.25604 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559bdf8bece0 con 0x559bdf84d3f0 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559bdf7ba0c0 con 0x559bdf84d3f0 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fea780bbc90 con 0x559bdf84d3f0 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 10 client.25604 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 20 client.25604 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf8bf330 0x559bdf8c1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 4 client.25604 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.861+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559bdf7385d0 con 0x559bdf8bf330 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.862+0000 7fea87400640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf8bf330 0x559bdf8c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.862+0000 7fea87400640 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf8bf330 0x559bdf8c1720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e6fd127248573d server_cookie=454e120ffbd0b650 in_seq=0 out_seq=0 2024-03-23T23:44:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.862+0000 7fea72000640 10 client.25604 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea72000640 1 -- 172.21.15.99:0/4265024106 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559bdf7385d0 con 0x559bdf8bf330 2024-03-23T23:44:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea72000640 10 client.25604 handle_command_reply: tid=0 2024-03-23T23:44:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 client.25604 shutdown 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 msgr2=0x559bdf85c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea780095c0 tx=0x7fea780092e0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 shutdown_connections 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559bdf8bf330 0x559bdf8c1720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bdf852f50 0x559bdf85d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559bdf84dde0 0x559bdf85c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea72000640 0 client.25604 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 --2- 172.21.15.99:0/4265024106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bdf84d3f0 0x559bdf85c290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 >> 172.21.15.99:0/4265024106 conn(0x559bdf848960 msgr2=0x559bdf8436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 shutdown_connections 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 1 -- 172.21.15.99:0/4265024106 wait complete. 2024-03-23T23:44:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.870+0000 7fea91a1b740 20 client.25604 trim_cache size 0 max 16384 2024-03-23T23:44:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:44:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:44:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:44:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:44:30.872 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 msgr2=0x7fea6c0a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 0x7fea6c0a4c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea8001b040 tx=0x7fea80000f40 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 msgr2=0x7fea88199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea78000c00 tx=0x7fea7802c650 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 shutdown_connections 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea6c0a2780 0x7fea6c0a4c40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea88101060 0x7fea881a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea881006c0 0x7fea88199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 --2- 172.21.15.99:0/228820523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880ffd60 0x7fea88199990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.871+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 >> 172.21.15.99:0/228820523 conn(0x7fea880fb980 msgr2=0x7fea880fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.872+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 shutdown_connections 2024-03-23T23:44:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:30.872+0000 7fea8d800640 1 -- 172.21.15.99:0/228820523 wait complete. 2024-03-23T23:44:32.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:31 smithi174 ceph-mon[37002]: pgmap v419: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-23T23:44:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:31 smithi099 ceph-mon[31715]: pgmap v419: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-23T23:44:32.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:31 smithi149 ceph-mon[36083]: pgmap v419: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-23T23:44:34.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:33 smithi174 ceph-mon[37002]: pgmap v420: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:34.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:33 smithi099 ceph-mon[31715]: pgmap v420: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:34.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:33 smithi149 ceph-mon[36083]: pgmap v420: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 10 op/s 2024-03-23T23:44:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:35 smithi174 ceph-mon[37002]: pgmap v421: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:35 smithi099 ceph-mon[31715]: pgmap v421: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:35 smithi149 ceph-mon[36083]: pgmap v421: 97 pgs: 97 active+clean; 671 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-23T23:44:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:37 smithi174 ceph-mon[37002]: pgmap v422: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:38.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:37 smithi149 ceph-mon[36083]: pgmap v422: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:38.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:37 smithi099 ceph-mon[31715]: pgmap v422: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-23T23:44:40.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:39 smithi174 ceph-mon[37002]: pgmap v423: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-23T23:44:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:39 smithi099 ceph-mon[31715]: pgmap v423: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-23T23:44:40.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:39 smithi149 ceph-mon[36083]: pgmap v423: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-23T23:44:42.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:41 smithi174 ceph-mon[37002]: pgmap v424: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-23T23:44:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:41 smithi099 ceph-mon[31715]: pgmap v424: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-23T23:44:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:41 smithi149 ceph-mon[36083]: pgmap v424: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-03-23T23:44:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 -- 172.21.15.99:0/1304190635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa80ffd60 msgr2=0x7fdfa8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 --2- 172.21.15.99:0/1304190635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa80ffd60 0x7fdfa8102190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802bc60 comp rx=0 tx=0).stop 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 -- 172.21.15.99:0/1304190635 shutdown_connections 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 --2- 172.21.15.99:0/1304190635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa8105080 0x7fdfa8105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 --2- 172.21.15.99:0/1304190635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa81026d0 0x7fdfa8104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 --2- 172.21.15.99:0/1304190635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa80ffd60 0x7fdfa8102190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.046+0000 7fdface00640 1 -- 172.21.15.99:0/1304190635 >> 172.21.15.99:0/1304190635 conn(0x7fdfa80fb980 msgr2=0x7fdfa80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.047+0000 7fdface00640 1 -- 172.21.15.99:0/1304190635 shutdown_connections 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.047+0000 7fdface00640 1 -- 172.21.15.99:0/1304190635 wait complete. 2024-03-23T23:44:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 Processor -- start 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 -- start start 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa80ffd60 0x7fdfa819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa8105080 0x7fdfa81a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8113790 con 0x7fdfa8105080 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8113610 con 0x7fdfa80ffd60 2024-03-23T23:44:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.048+0000 7fdface00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa8113910 con 0x7fdfa81026d0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa80ffd60 0x7fdfa819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57160/0 (socket says 172.21.15.99:57160) 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 -- 172.21.15.99:0/3021223356 learned_addr learned my addr 172.21.15.99:0/3021223356 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa8105080 0x7fdfa81a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 -- 172.21.15.99:0/3021223356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa80ffd60 msgr2=0x7fdfa819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa80ffd60 0x7fdfa819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 -- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa8105080 msgr2=0x7fdfa81a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa8105080 0x7fdfa81a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 -- 172.21.15.99:0/3021223356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa81a59d0 con 0x7fdfa81026d0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.049+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c920 tx=0x7fdf9400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.050+0000 7fdf9fe00640 1 -- 172.21.15.99:0/3021223356 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94007c70 con 0x7fdfa81026d0 2024-03-23T23:44:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.050+0000 7fdf9fe00640 1 -- 172.21.15.99:0/3021223356 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf94007e10 con 0x7fdfa81026d0 2024-03-23T23:44:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.050+0000 7fdf9fe00640 1 -- 172.21.15.99:0/3021223356 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94018890 con 0x7fdfa81026d0 2024-03-23T23:44:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.050+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa81a5cc0 con 0x7fdfa81026d0 2024-03-23T23:44:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.050+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa81a6190 con 0x7fdfa81026d0 2024-03-23T23:44:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.055+0000 7fdfb1081740 1 Processor -- start 2024-03-23T23:44:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.055+0000 7fdfb1081740 1 -- start start 2024-03-23T23:44:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 0x559c655d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655d2f50 0x559c655d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c655903b0 con 0x559c655d2f50 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c65532830 con 0x559c655cd3f0 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfb1081740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c654db5d0 con 0x559c655cdde0 2024-03-23T23:44:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 0x559c655d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 0x559c655d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57170/0 (socket says 172.21.15.99:57170) 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfa7400640 1 -- 172.21.15.99:0/2357886688 learned_addr learned my addr 172.21.15.99:0/2357886688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.056+0000 7fdf9fe00640 1 -- 172.21.15.99:0/3021223356 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdf94018a70 con 0x7fdfa81026d0 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 -- 172.21.15.99:0/2357886688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 msgr2=0x559c655d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 0x559c655d28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 -- 172.21.15.99:0/2357886688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655d2f50 msgr2=0x559c655d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655d2f50 0x559c655d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 -- 172.21.15.99:0/2357886688 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c654f32c0 con 0x559c655cd3f0 2024-03-23T23:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655cd810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009dd0 tx=0x7fdfa000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6f33bec9a2d3c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdf9cc00640 1 -- 172.21.15.99:0/2357886688 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0004b20 con 0x559c655cd3f0 2024-03-23T23:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdf9cc00640 1 -- 172.21.15.99:0/2357886688 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0017070 con 0x559c655cd3f0 2024-03-23T23:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.057+0000 7fdf9cc00640 1 -- 172.21.15.99:0/2357886688 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0011790 con 0x559c655cd3f0 2024-03-23T23:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.058+0000 7fdfb1081740 1 -- 172.21.15.99:0/2357886688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 msgr2=0x559c655cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.058+0000 7fdfb1081740 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655cd810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009dd0 tx=0x7fdfa000f730 comp rx=0 tx=0).stop 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.058+0000 7fdf9fe00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 0x7fdf8c0a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfb1081740 1 -- 172.21.15.99:0/2357886688 shutdown_connections 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfb1081740 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655d2f50 0x559c655d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfb1081740 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cdde0 0x559c655d28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfb1081740 1 --2- 172.21.15.99:0/2357886688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655cd810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdf9fe00640 1 -- 172.21.15.99:0/3021223356 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdf940cecc0 con 0x7fdfa81026d0 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfb1081740 1 -- 172.21.15.99:0/2357886688 >> 172.21.15.99:0/2357886688 conn(0x559c655c8960 msgr2=0x559c655c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.059+0000 7fdfa7400640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 0x7fdf8c0a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.060+0000 7fdfa7400640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 0x7fdf8c0a4cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559c655ce1e0 tx=0x7fdf98004c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.060+0000 7fdfb1081740 1 -- 172.21.15.99:0/2357886688 shutdown_connections 2024-03-23T23:44:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.061+0000 7fdfb1081740 1 -- 172.21.15.99:0/2357886688 wait complete. 2024-03-23T23:44:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.061+0000 7fdfb1081740 1 -- start start 2024-03-23T23:44:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 Processor -- start 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c654eea20 con 0x559c655cdde0 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6552e8d0 con 0x559c655d2f50 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c655d5f00 con 0x559c655cd3f0 2024-03-23T23:44:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfb1081740 10 client.0 _ll_register_callbacks cb 0x559c6552b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59730/0 (socket says 172.21.15.99:59730) 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.062+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 -- 172.21.15.99:0/3791640996 learned_addr learned my addr 172.21.15.99:0/3791640996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 -- 172.21.15.99:0/3791640996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 msgr2=0x559c655d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 -- 172.21.15.99:0/3791640996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 msgr2=0x559c655d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 -- 172.21.15.99:0/3791640996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c65247210 con 0x559c655d2f50 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.063+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0000f80 tx=0x7fdfa0004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76853f6471f6825e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdf91600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa002b060 con 0x559c655d2f50 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdfa000fe40 con 0x559c655d2f50 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfa0017070 con 0x559c655d2f50 2024-03-23T23:44:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfa0005210 con 0x559c655d2f50 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdfb1081740 10 client.25613 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.064+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559c65313620 con 0x559c655d2f50 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.065+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0017070 con 0x559c655d2f50 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fdfa00114b0 con 0x559c655d2f50 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 10 client.25613 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 20 client.25613 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c655e31c0 0x559c655e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 4 client.25613 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c6553fd70 con 0x559c655e31c0 2024-03-23T23:44:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.066+0000 7fdfa7400640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c655e31c0 0x559c655e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.067+0000 7fdf91600640 10 client.25613 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.067+0000 7fdfa7400640 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c655e31c0 0x559c655e55b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d686ba90f485252 server_cookie=d28d69fd3a20c43e in_seq=0 out_seq=0 2024-03-23T23:44:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdf91600640 1 -- 172.21.15.99:0/3791640996 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559c6553fd70 con 0x559c655e31c0 2024-03-23T23:44:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdf91600640 10 client.25613 handle_command_reply: tid=0 2024-03-23T23:44:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 client.25613 shutdown 2024-03-23T23:44:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 msgr2=0x559c655deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0000f80 tx=0x7fdfa0004c70 comp rx=0 tx=0).stop 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 shutdown_connections 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c655e31c0 0x559c655e55b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655deb80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 --2- 172.21.15.99:0/3791640996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 >> 172.21.15.99:0/3791640996 conn(0x559c655c8960 msgr2=0x559c655c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.068+0000 7fdf91600640 0 client.25613 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.069+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 shutdown_connections 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.069+0000 7fdfb1081740 1 -- 172.21.15.99:0/3791640996 wait complete. 2024-03-23T23:44:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.069+0000 7fdfb1081740 20 client.25613 trim_cache size 0 max 16384 2024-03-23T23:44:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.083+0000 7fdfb1081740 1 Processor -- start 2024-03-23T23:44:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.083+0000 7fdfb1081740 1 -- start start 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81aede0 con 0x559c655cdde0 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81b0760 con 0x559c655d2f50 2024-03-23T23:44:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfb1081740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81af440 con 0x559c655cd3f0 2024-03-23T23:44:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57198/0 (socket says 172.21.15.99:57198) 2024-03-23T23:44:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfa7400640 1 -- 172.21.15.99:0/303309386 learned_addr learned my addr 172.21.15.99:0/303309386 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.084+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 -- 172.21.15.99:0/303309386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 msgr2=0x559c655cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 -- 172.21.15.99:0/303309386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 msgr2=0x559c655deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 -- 172.21.15.99:0/303309386 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c655d23f0 con 0x559c655cd3f0 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.085+0000 7fdfa7400640 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802c1a0 tx=0x7fdf98045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453ddc6d2baecc21 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdf93e00640 1 -- 172.21.15.99:0/303309386 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980552c0 con 0x559c655cd3f0 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdf93e00640 1 -- 172.21.15.99:0/303309386 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf9802dda0 con 0x559c655cd3f0 2024-03-23T23:44:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdf93e00640 1 -- 172.21.15.99:0/303309386 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98055d90 con 0x559c655cd3f0 2024-03-23T23:44:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 -- 172.21.15.99:0/303309386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 msgr2=0x559c655c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802c1a0 tx=0x7fdf98045000 comp rx=0 tx=0).stop 2024-03-23T23:44:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 -- 172.21.15.99:0/303309386 shutdown_connections 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655d2f50 0x559c655cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 --2- 172.21.15.99:0/303309386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655cd3f0 0x559c655c8d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.086+0000 7fdfb1081740 1 -- 172.21.15.99:0/303309386 >> 172.21.15.99:0/303309386 conn(0x559c655c8960 msgr2=0x559c655d6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.087+0000 7fdfb1081740 1 -- 172.21.15.99:0/303309386 shutdown_connections 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.087+0000 7fdfb1081740 1 -- 172.21.15.99:0/303309386 wait complete. 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.087+0000 7fdfb1081740 1 -- start start 2024-03-23T23:44:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.087+0000 7fdfb1081740 1 Processor -- start 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655dc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 0x559c655dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81b0100 con 0x559c655cdde0 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81afaa0 con 0x559c655cd3f0 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa81acfa0 con 0x559c655d2f50 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfb1081740 10 client.0 _ll_register_callbacks cb 0x7fdfa0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59754/0 (socket says 172.21.15.99:59754) 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 -- 172.21.15.99:0/995998301 learned_addr learned my addr 172.21.15.99:0/995998301 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 0x559c655dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 -- 172.21.15.99:0/995998301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 msgr2=0x559c655dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 0x559c655dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.088+0000 7fdfa7400640 1 -- 172.21.15.99:0/995998301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 msgr2=0x559c655dc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655dc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa7400640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655dc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa7400640 1 -- 172.21.15.99:0/995998301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c655ddd20 con 0x559c655cd3f0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa7e00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 0x559c655dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdfa7400640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005c40 tx=0x7fdf98004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86d134ac06a111c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdf91600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98039040 con 0x559c655cd3f0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.089+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdf9805a040 con 0x559c655cd3f0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf9802dda0 con 0x559c655cd3f0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf98005420 con 0x559c655cd3f0 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdfb1081740 10 client.25619 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559c6563ea80 con 0x559c655cd3f0 2024-03-23T23:44:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf9802dda0 con 0x559c655cd3f0 2024-03-23T23:44:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.090+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fdf98034880 con 0x559c655cd3f0 2024-03-23T23:44:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 10 client.25619 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 20 client.25619 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c6563f240 0x559c65641630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 4 client.25619 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c6553a0c0 con 0x559c6563f240 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.091+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c6563f240 0x559c65641630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.092+0000 7fdfa6a00640 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c6563f240 0x559c65641630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7cf0cf5f5c564 server_cookie=93fa3c1d531adc5c in_seq=0 out_seq=0 2024-03-23T23:44:44.093 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.092+0000 7fdf91600640 10 client.25619 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.095+0000 7fdf91600640 1 -- 172.21.15.99:0/995998301 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c6553a0c0 con 0x559c6563f240 2024-03-23T23:44:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.095+0000 7fdf91600640 10 client.25619 handle_command_reply: tid=0 2024-03-23T23:44:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.095+0000 7fdfb1081740 1 client.25619 shutdown 2024-03-23T23:44:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 msgr2=0x559c655dc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005c40 tx=0x7fdf98004a20 comp rx=0 tx=0).stop 2024-03-23T23:44:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 shutdown_connections 2024-03-23T23:44:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x559c6563f240 0x559c65641630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c655d2f50 0x559c655dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c655cdde0 0x559c655dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 --2- 172.21.15.99:0/995998301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c655cd3f0 0x559c655dc1e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 >> 172.21.15.99:0/995998301 conn(0x559c655c8960 msgr2=0x559c655c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdf91600640 0 client.25619 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 shutdown_connections 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 1 -- 172.21.15.99:0/995998301 wait complete. 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.096+0000 7fdfb1081740 20 client.25619 trim_cache size 0 max 16384 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:44:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:44:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:44:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 msgr2=0x7fdf8c0a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 0x7fdf8c0a4cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559c655ce1e0 tx=0x7fdf98004c90 comp rx=0 tx=0).stop 2024-03-23T23:44:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 msgr2=0x7fdfa819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:44.098 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400c920 tx=0x7fdf9400cde0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 shutdown_connections 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fdf8c0a27f0 0x7fdf8c0a4cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa8105080 0x7fdfa81a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdfa81026d0 0x7fdfa819e300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 --2- 172.21.15.99:0/3021223356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfa80ffd60 0x7fdfa819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.097+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 >> 172.21.15.99:0/3021223356 conn(0x7fdfa80fb980 msgr2=0x7fdfa8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.098+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 shutdown_connections 2024-03-23T23:44:44.099 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:44.098+0000 7fdface00640 1 -- 172.21.15.99:0/3021223356 wait complete. 2024-03-23T23:44:44.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:43 smithi174 ceph-mon[37002]: pgmap v425: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T23:44:44.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:43 smithi099 ceph-mon[31715]: pgmap v425: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T23:44:44.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:43 smithi149 ceph-mon[36083]: pgmap v425: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-23T23:44:46.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:45 smithi174 ceph-mon[37002]: pgmap v426: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-23T23:44:46.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:45 smithi099 ceph-mon[31715]: pgmap v426: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-23T23:44:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:45 smithi149 ceph-mon[36083]: pgmap v426: 97 pgs: 97 active+clean; 672 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-23T23:44:48.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:47 smithi174 ceph-mon[37002]: pgmap v427: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:44:48.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:47 smithi099 ceph-mon[31715]: pgmap v427: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:44:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:47 smithi149 ceph-mon[36083]: pgmap v427: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-23T23:44:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:48 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:44:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:48 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:49.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:48 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:48 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:44:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:48 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:49.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:48 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:49.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:48 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:44:49.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:48 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:49.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:48 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:49 smithi099 ceph-mon[31715]: pgmap v428: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:44:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:44:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:49 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:49 smithi149 ceph-mon[36083]: pgmap v428: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:44:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:44:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:49 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:49 smithi174 ceph-mon[37002]: pgmap v428: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:44:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:44:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:49 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:44:52.225 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:51 smithi174 ceph-mon[37002]: pgmap v429: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:51 smithi099 ceph-mon[31715]: pgmap v429: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:52.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:51 smithi149 ceph-mon[36083]: pgmap v429: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:54.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:53 smithi099 ceph-mon[31715]: pgmap v430: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:44:54.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:53 smithi149 ceph-mon[36083]: pgmap v430: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:44:54.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:53 smithi174 ceph-mon[37002]: pgmap v430: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-23T23:44:55.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:44:55.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:44:55.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 -- 172.21.15.99:0/1139322259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328102000 msgr2=0x7f93281043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 --2- 172.21.15.99:0/1139322259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328102000 0x7f93281043f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9318009900 tx=0x7f931802bc40 comp rx=0 tx=0).stop 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 -- 172.21.15.99:0/1139322259 shutdown_connections 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 --2- 172.21.15.99:0/1139322259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328107260 0x7f93281096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 --2- 172.21.15.99:0/1139322259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328104930 0x7f9328106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 --2- 172.21.15.99:0/1139322259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328102000 0x7f93281043f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 -- 172.21.15.99:0/1139322259 >> 172.21.15.99:0/1139322259 conn(0x7f93280fba10 msgr2=0x7f93280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 -- 172.21.15.99:0/1139322259 shutdown_connections 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.641+0000 7f932d000640 1 -- 172.21.15.99:0/1139322259 wait complete. 2024-03-23T23:44:55.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.642+0000 7f932d000640 1 Processor -- start 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.642+0000 7f932d000640 1 -- start start 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328104930 0x7f93281967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 0x7f9328192e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932810d260 con 0x7f9328104930 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932810d0e0 con 0x7f9328102000 2024-03-23T23:44:55.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f932d000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932810d3e0 con 0x7f9328107260 2024-03-23T23:44:55.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f9327400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f9327400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38432/0 (socket says 172.21.15.99:38432) 2024-03-23T23:44:55.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f9327400640 1 -- 172.21.15.99:0/36922955 learned_addr learned my addr 172.21.15.99:0/36922955 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.643+0000 7f9327e00640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 0x7f9328192e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9326a00640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328104930 0x7f93281967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 -- 172.21.15.99:0/36922955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 msgr2=0x7f9328192e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 0x7f9328192e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 -- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328104930 msgr2=0x7f93281967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328104930 0x7f93281967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 -- 172.21.15.99:0/36922955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93281936e0 con 0x7f9328102000 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327e00640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 0x7f9328192e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.644+0000 7f9327400640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f931802c8f0 tx=0x7f9318005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.645+0000 7f931fe00640 1 -- 172.21.15.99:0/36922955 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318041070 con 0x7f9328102000 2024-03-23T23:44:55.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.645+0000 7f931fe00640 1 -- 172.21.15.99:0/36922955 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f93180044d0 con 0x7f9328102000 2024-03-23T23:44:55.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.645+0000 7f931fe00640 1 -- 172.21.15.99:0/36922955 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318034440 con 0x7f9328102000 2024-03-23T23:44:55.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.645+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93281a5f00 con 0x7f9328102000 2024-03-23T23:44:55.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.645+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93281a62b0 con 0x7f9328102000 2024-03-23T23:44:55.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 Processor -- start 2024-03-23T23:44:55.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 -- start start 2024-03-23T23:44:55.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f931fe00640 1 -- 172.21.15.99:0/36922955 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9318044070 con 0x7f9328102000 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc383a250 con 0x55abc38dabb0 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc37e2a30 con 0x55abc38d5050 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.650+0000 7f933127e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc38362f0 con 0x55abc38d5a40 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.651+0000 7f9327e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.651+0000 7f9327e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38448/0 (socket says 172.21.15.99:38448) 2024-03-23T23:44:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.651+0000 7f9327e00640 1 -- 172.21.15.99:0/2395926820 learned_addr learned my addr 172.21.15.99:0/2395926820 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.651+0000 7f9327400640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9326a00640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 -- 172.21.15.99:0/2395926820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 msgr2=0x55abc38da540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38da540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 -- 172.21.15.99:0/2395926820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 msgr2=0x55abc38dd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38dd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 -- 172.21.15.99:0/2395926820 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc37cd690 con 0x55abc38d5050 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327400640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.652+0000 7f9327e00640 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f932000af40 tx=0x7f932000e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0417263b0a33b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.653+0000 7f931cc00640 1 -- 172.21.15.99:0/2395926820 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320003b70 con 0x55abc38d5050 2024-03-23T23:44:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.653+0000 7f931cc00640 1 -- 172.21.15.99:0/2395926820 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9320003d10 con 0x55abc38d5050 2024-03-23T23:44:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.653+0000 7f931cc00640 1 -- 172.21.15.99:0/2395926820 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320010c50 con 0x55abc38d5050 2024-03-23T23:44:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.654+0000 7f931fe00640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 0x7f930c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.654+0000 7f931fe00640 1 -- 172.21.15.99:0/36922955 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93180f0720 con 0x7f9328102000 2024-03-23T23:44:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.655+0000 7f9327400640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 0x7f930c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.655+0000 7f933127e740 1 -- 172.21.15.99:0/2395926820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 msgr2=0x55abc38d5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.655+0000 7f933127e740 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f932000af40 tx=0x7f932000e7e0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f9327400640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 0x7f930c0a4dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f93180bcfd0 tx=0x7f93180dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 -- 172.21.15.99:0/2395926820 shutdown_connections 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38dd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38da540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 --2- 172.21.15.99:0/2395926820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38d5470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 -- 172.21.15.99:0/2395926820 >> 172.21.15.99:0/2395926820 conn(0x55abc389d200 msgr2=0x55abc38d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 -- 172.21.15.99:0/2395926820 shutdown_connections 2024-03-23T23:44:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 -- 172.21.15.99:0/2395926820 wait complete. 2024-03-23T23:44:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.656+0000 7f933127e740 1 -- start start 2024-03-23T23:44:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 Processor -- start 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc38eeb70 con 0x55abc38dabb0 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc37f5e80 con 0x55abc38d5050 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc38eecf0 con 0x55abc38d5a40 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.657+0000 7f933127e740 10 client.0 _ll_register_callbacks cb 0x55abc3808130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.658+0000 7f9326a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.658+0000 7f9326a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48448/0 (socket says 172.21.15.99:48448) 2024-03-23T23:44:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.658+0000 7f9326a00640 1 -- 172.21.15.99:0/484833438 learned_addr learned my addr 172.21.15.99:0/484833438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.658+0000 7f9327e00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9327e00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 -- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 msgr2=0x55abc38f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 -- 172.21.15.99:0/484833438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 msgr2=0x55abc38f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 -- 172.21.15.99:0/484833438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abc36187d0 con 0x55abc38d5a40 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9327e00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9326a00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9314013710 tx=0x7f9314013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=776913be497bf34c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9311600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:55.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.659+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314011d70 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9314004030 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9320003d10 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9314004230 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f933127e740 10 client.25660 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abc354e3f0 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.660+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9320003d10 con 0x55abc38d5a40 2024-03-23T23:44:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f931401a070 con 0x55abc38d5a40 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 10 client.25660 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 20 client.25660 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc38fbf40 0x55abc38fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 4 client.25660 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.661+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abc3847790 con 0x55abc38fbf40 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.662+0000 7f9327e00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc38fbf40 0x55abc38fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.662+0000 7f9327e00640 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc38fbf40 0x55abc38fe360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6793bf068b7ad50 server_cookie=daea28a09a7d9295 in_seq=0 out_seq=0 2024-03-23T23:44:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.662+0000 7f9311600640 10 client.25660 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.663+0000 7f9311600640 1 -- 172.21.15.99:0/484833438 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55abc3847790 con 0x55abc38fbf40 2024-03-23T23:44:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.663+0000 7f9311600640 10 client.25660 handle_command_reply: tid=0 2024-03-23T23:44:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.663+0000 7f933127e740 1 client.25660 shutdown 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 msgr2=0x55abc38f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9314013710 tx=0x7f9314013bd0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 shutdown_connections 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc38fbf40 0x55abc38fe360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38dabb0 0x55abc38f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38d5a40 0x55abc38f0600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 --2- 172.21.15.99:0/484833438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 >> 172.21.15.99:0/484833438 conn(0x55abc389d200 msgr2=0x55abc38cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f9311600640 0 client.25660 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 shutdown_connections 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 1 -- 172.21.15.99:0/484833438 wait complete. 2024-03-23T23:44:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.664+0000 7f933127e740 20 client.25660 trim_cache size 0 max 16384 2024-03-23T23:44:55.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.681+0000 7f933127e740 1 Processor -- start 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.681+0000 7f933127e740 1 -- start start 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5050 0x55abc38da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a6610 con 0x55abc38d5050 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a7f90 con 0x55abc38d5a40 2024-03-23T23:44:55.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f933127e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a6c70 con 0x55abc38dabb0 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f9326a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f9326a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38480/0 (socket says 172.21.15.99:38480) 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f9326a00640 1 -- 172.21.15.99:0/3238114609 learned_addr learned my addr 172.21.15.99:0/3238114609 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f9327e00640 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.682+0000 7f9327e00640 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5050 0x55abc38da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 -- 172.21.15.99:0/3238114609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 msgr2=0x55abc38d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38d6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 -- 172.21.15.99:0/3238114609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5050 msgr2=0x55abc38da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5050 0x55abc38da060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 -- 172.21.15.99:0/3238114609 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc350e0f0 con 0x55abc38d5a40 2024-03-23T23:44:55.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.683+0000 7f9326a00640 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f931400f690 tx=0x7f931400f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31952a1fa9ad01c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.684+0000 7f9313e00640 1 -- 172.21.15.99:0/3238114609 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400fbe0 con 0x55abc38d5a40 2024-03-23T23:44:55.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.684+0000 7f9313e00640 1 -- 172.21.15.99:0/3238114609 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f931400fd80 con 0x55abc38d5a40 2024-03-23T23:44:55.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.684+0000 7f9313e00640 1 -- 172.21.15.99:0/3238114609 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401c920 con 0x55abc38d5a40 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.684+0000 7f933127e740 1 -- 172.21.15.99:0/3238114609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 msgr2=0x55abc38f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.684+0000 7f933127e740 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f931400f690 tx=0x7f931400f9b0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 -- 172.21.15.99:0/3238114609 shutdown_connections 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5a40 0x55abc38f7600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 --2- 172.21.15.99:0/3238114609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5050 0x55abc38da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 -- 172.21.15.99:0/3238114609 >> 172.21.15.99:0/3238114609 conn(0x55abc389d200 msgr2=0x55abc38d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 -- 172.21.15.99:0/3238114609 shutdown_connections 2024-03-23T23:44:55.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 -- 172.21.15.99:0/3238114609 wait complete. 2024-03-23T23:44:55.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 -- start start 2024-03-23T23:44:55.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.685+0000 7f933127e740 1 Processor -- start 2024-03-23T23:44:55.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5a40 0x55abc38f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a7930 con 0x55abc38d5a40 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a72d0 con 0x55abc38d5050 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a0230 con 0x55abc38dabb0 2024-03-23T23:44:55.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.686+0000 7f933127e740 10 client.0 _ll_register_callbacks cb 0x7f9314009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38482/0 (socket says 172.21.15.99:38482) 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 -- 172.21.15.99:0/2466710939 learned_addr learned my addr 172.21.15.99:0/2466710939 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9326a00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5a40 0x55abc38f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 -- 172.21.15.99:0/2466710939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 msgr2=0x55abc38f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 -- 172.21.15.99:0/2466710939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5a40 msgr2=0x55abc38f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5a40 0x55abc38f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.687+0000 7f9327e00640 1 -- 172.21.15.99:0/2466710939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abc38f6d00 con 0x55abc38d5050 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.688+0000 7f9327e00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f932000a750 tx=0x7f932000e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefd4c0db326abae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.688+0000 7f9311600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.688+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320029920 con 0x55abc38d5050 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.688+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9320029ac0 con 0x55abc38d5050 2024-03-23T23:44:55.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.688+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f931400fd80 con 0x55abc38d5050 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9320029cc0 con 0x55abc38d5050 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 10 client.25637 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abc3957c80 con 0x55abc38d5050 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f931400fd80 con 0x55abc38d5050 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f932001a5f0 con 0x55abc38d5050 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 10 client.25637 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 20 client.25637 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc3958440 0x55abc395a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 4 client.25637 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:44:55.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.689+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55abc3841ae0 con 0x55abc3958440 2024-03-23T23:44:55.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f9326a00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc3958440 0x55abc395a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f9326a00640 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc3958440 0x55abc395a860 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7fa08d420f3c73 server_cookie=97dfbd92e4b16cd2 in_seq=0 out_seq=0 2024-03-23T23:44:55.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f9311600640 10 client.25637 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f9311600640 1 -- 172.21.15.99:0/2466710939 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55abc3841ae0 con 0x55abc3958440 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f9311600640 10 client.25637 handle_command_reply: tid=0 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.690+0000 7f933127e740 1 client.25637 shutdown 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 msgr2=0x55abc38f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f932000a750 tx=0x7f932000e7e0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 shutdown_connections 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55abc3958440 0x55abc395a860 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc38dabb0 0x55abc38f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55abc38d5a40 0x55abc38f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 --2- 172.21.15.99:0/2466710939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abc38d5050 0x55abc38f51c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 >> 172.21.15.99:0/2466710939 conn(0x55abc389d200 msgr2=0x55abc38d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f9311600640 0 client.25637 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 shutdown_connections 2024-03-23T23:44:55.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 1 -- 172.21.15.99:0/2466710939 wait complete. 2024-03-23T23:44:55.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.691+0000 7f933127e740 20 client.25637 trim_cache size 0 max 16384 2024-03-23T23:44:55.696 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:44:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-23T23:44:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-23T23:44:55.696 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.692+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 msgr2=0x7f930c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.692+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 0x7f930c0a4dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f93180bcfd0 tx=0x7f93180dd040 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.692+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 msgr2=0x7f93281962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.692+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f931802c8f0 tx=0x7f9318005d50 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 shutdown_connections 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f930c0a2900 0x7f930c0a4dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9328107260 0x7f9328192e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9328104930 0x7f93281967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 --2- 172.21.15.99:0/36922955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9328102000 0x7f93281962b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 >> 172.21.15.99:0/36922955 conn(0x7f93280fba10 msgr2=0x7f9328101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 shutdown_connections 2024-03-23T23:44:55.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.693+0000 7f932d000640 1 -- 172.21.15.99:0/36922955 wait complete. 2024-03-23T23:44:55.711 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T23:44:55.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.983+0000 7fa6a2c00640 1 -- 172.21.15.99:0/3092592368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1048a0 msgr2=0x7fa69c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.983+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/3092592368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1048a0 0x7fa69c106c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa688009860 tx=0x7fa68802bb90 comp rx=0 tx=0).stop 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 -- 172.21.15.99:0/3092592368 shutdown_connections 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/3092592368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1071d0 0x7fa69c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/3092592368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1048a0 0x7fa69c106c90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/3092592368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 0x7fa69c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 -- 172.21.15.99:0/3092592368 >> 172.21.15.99:0/3092592368 conn(0x7fa69c0fb980 msgr2=0x7fa69c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 -- 172.21.15.99:0/3092592368 shutdown_connections 2024-03-23T23:44:55.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.984+0000 7fa6a2c00640 1 -- 172.21.15.99:0/3092592368 wait complete. 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.985+0000 7fa6a2c00640 1 Processor -- start 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 -- start start 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 0x7fa69c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1071d0 0x7fa69c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c10d1e0 con 0x7fa69c101f70 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c10d060 con 0x7fa69c1071d0 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.986+0000 7fa6a2c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c10d360 con 0x7fa69c1048a0 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.987+0000 7fa6a0e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.987+0000 7fa6a0e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48480/0 (socket says 172.21.15.99:48480) 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.987+0000 7fa6a0e00640 1 -- 172.21.15.99:0/2458257138 learned_addr learned my addr 172.21.15.99:0/2458257138 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.987+0000 7fa6a1800640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 0x7fa69c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.988+0000 7fa6a0e00640 1 -- 172.21.15.99:0/2458257138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1071d0 msgr2=0x7fa69c19ee10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.988+0000 7fa6a0e00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1071d0 0x7fa69c19ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.988+0000 7fa6a0e00640 1 -- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 msgr2=0x7fa69c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.988+0000 7fa6a0e00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 0x7fa69c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.988+0000 7fa6a0e00640 1 -- 172.21.15.99:0/2458257138 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c0ffe60 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa6a0e00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa68802c1a0 tx=0x7fa6880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688041020 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6880323b0 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c1000f0 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688039040 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.989+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c1006b0 con 0x7fa69c1048a0 2024-03-23T23:44:55.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.991+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c101b50 con 0x7fa69c1048a0 2024-03-23T23:44:55.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.994+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa688044020 con 0x7fa69c1048a0 2024-03-23T23:44:55.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.996+0000 7fa69a000640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 0x7fa6840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:55.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.997+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6880f03a0 con 0x7fa69c1048a0 2024-03-23T23:44:55.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.997+0000 7fa6a1800640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 0x7fa6840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:55.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.998+0000 7fa6a1800640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 0x7fa6840a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa690009530 tx=0x7fa690009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:55.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:55.999+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa68804b020 con 0x7fa69c1048a0 2024-03-23T23:44:56.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:55 smithi099 ceph-mon[31715]: pgmap v431: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:55 smithi149 ceph-mon[36083]: pgmap v431: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:56.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:55 smithi174 ceph-mon[37002]: pgmap v431: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:56.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.448+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa69c0fd290 con 0x7fa69c1048a0 2024-03-23T23:44:56.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.449+0000 7fa69a000640 1 -- 172.21.15.99:0/2458257138 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fa6880bd3c0 con 0x7fa69c1048a0 2024-03-23T23:44:56.450 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:44:56.450 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:44:56.450 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:44:56.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 msgr2=0x7fa6840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 0x7fa6840a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa690009530 tx=0x7fa690009260 comp rx=0 tx=0).stop 2024-03-23T23:44:56.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 msgr2=0x7fa69c197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa68802c1a0 tx=0x7fa6880042f0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 shutdown_connections 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fa6840a2880 0x7fa6840a4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa69c1071d0 0x7fa69c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa69c1048a0 0x7fa69c197da0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 --2- 172.21.15.99:0/2458257138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa69c101f70 0x7fa69c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 >> 172.21.15.99:0/2458257138 conn(0x7fa69c0fb980 msgr2=0x7fa69c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 shutdown_connections 2024-03-23T23:44:56.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.454+0000 7fa6a2c00640 1 -- 172.21.15.99:0/2458257138 wait complete. 2024-03-23T23:44:56.480 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:56.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T23:44:56.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 -- 172.21.15.99:0/331880581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 msgr2=0x7fb97c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 --2- 172.21.15.99:0/331880581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c1043f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009900 tx=0x7fb96c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 -- 172.21.15.99:0/331880581 shutdown_connections 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 --2- 172.21.15.99:0/331880581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 0x7fb97c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 --2- 172.21.15.99:0/331880581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 0x7fb97c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 --2- 172.21.15.99:0/331880581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c1043f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 -- 172.21.15.99:0/331880581 >> 172.21.15.99:0/331880581 conn(0x7fb97c0fba10 msgr2=0x7fb97c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 -- 172.21.15.99:0/331880581 shutdown_connections 2024-03-23T23:44:56.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.680+0000 7fb97b200640 1 -- 172.21.15.99:0/331880581 wait complete. 2024-03-23T23:44:56.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.681+0000 7fb97b200640 1 Processor -- start 2024-03-23T23:44:56.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 -- start start 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 0x7fb97c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 0x7fb97c1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c10d2c0 con 0x7fb97c107260 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c10d140 con 0x7fb97c104930 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.682+0000 7fb97b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c10d440 con 0x7fb97c102000 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb973e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 0x7fb97c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48510/0 (socket says 172.21.15.99:48510) 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 -- 172.21.15.99:0/3920379972 learned_addr learned my addr 172.21.15.99:0/3920379972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb97a800640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 0x7fb97c1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 -- 172.21.15.99:0/3920379972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 msgr2=0x7fb97c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 0x7fb97c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 -- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 msgr2=0x7fb97c1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.683+0000 7fb979e00640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 0x7fb97c1a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.684+0000 7fb979e00640 1 -- 172.21.15.99:0/3920379972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c1a1730 con 0x7fb97c102000 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.684+0000 7fb979e00640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000c00 tx=0x7fb96c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.685+0000 7fb972a00640 1 -- 172.21.15.99:0/3920379972 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c043020 con 0x7fb97c102000 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.685+0000 7fb972a00640 1 -- 172.21.15.99:0/3920379972 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb96c0054e0 con 0x7fb97c102000 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.685+0000 7fb972a00640 1 -- 172.21.15.99:0/3920379972 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c003df0 con 0x7fb97c102000 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.685+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c1a19c0 con 0x7fb97c102000 2024-03-23T23:44:56.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.685+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c1a1ec0 con 0x7fb97c102000 2024-03-23T23:44:56.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.690+0000 7fb983f9e740 1 Processor -- start 2024-03-23T23:44:56.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.690+0000 7fb983f9e740 1 -- start start 2024-03-23T23:44:56.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.690+0000 7fb972a00640 1 -- 172.21.15.99:0/3920379972 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb96c046020 con 0x7fb97c102000 2024-03-23T23:44:56.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c381201850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381208620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb97a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb97a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48526/0 (socket says 172.21.15.99:48526) 2024-03-23T23:44:56.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38115c2c0 con 0x55c3812013d0 2024-03-23T23:44:56.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c381104c90 con 0x55c381201d90 2024-03-23T23:44:56.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.691+0000 7fb983f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c381158360 con 0x55c381200200 2024-03-23T23:44:56.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 -- 172.21.15.99:0/1699629890 learned_addr learned my addr 172.21.15.99:0/1699629890 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb979e00640 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c381201850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb972a00640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 0x7fb9600a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 -- 172.21.15.99:0/1699629890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 msgr2=0x55c381208620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381208620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 -- 172.21.15.99:0/1699629890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 msgr2=0x55c381201850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c381201850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.692+0000 7fb97a800640 1 -- 172.21.15.99:0/1699629890 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c381129080 con 0x55c381200200 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.693+0000 7fb972a00640 1 -- 172.21.15.99:0/3920379972 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb96c0f0dd0 con 0x7fb97c102000 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.693+0000 7fb97a800640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 0x7fb9600a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.693+0000 7fb97a800640 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb974009f20 tx=0x7fb97400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bde35a223ec06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.694+0000 7fb96b400640 1 -- 172.21.15.99:0/1699629890 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974007d50 con 0x55c381200200 2024-03-23T23:44:56.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.694+0000 7fb96b400640 1 -- 172.21.15.99:0/1699629890 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9740178e0 con 0x55c381200200 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.694+0000 7fb96b400640 1 -- 172.21.15.99:0/1699629890 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400a710 con 0x55c381200200 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.694+0000 7fb97a800640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 0x7fb9600a4e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb97401f000 tx=0x7fb974020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 -- 172.21.15.99:0/1699629890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 msgr2=0x55c381200620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb974009f20 tx=0x7fb97400c8b0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 -- 172.21.15.99:0/1699629890 shutdown_connections 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381208620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c381201850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 --2- 172.21.15.99:0/1699629890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c381200620 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 -- 172.21.15.99:0/1699629890 >> 172.21.15.99:0/1699629890 conn(0x55c3811fb5f0 msgr2=0x55c3811fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 -- 172.21.15.99:0/1699629890 shutdown_connections 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.695+0000 7fb983f9e740 1 -- 172.21.15.99:0/1699629890 wait complete. 2024-03-23T23:44:56.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.696+0000 7fb983f9e740 1 -- start start 2024-03-23T23:44:56.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.696+0000 7fb983f9e740 1 Processor -- start 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.696+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c38120a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c381211a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c381208da0 con 0x55c381201d90 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3811180d0 con 0x55c381200200 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c381208f20 con 0x55c3812013d0 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb983f9e740 10 client.0 _ll_register_callbacks cb 0x55c38111ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb979e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb973e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c38120a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb979e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38534/0 (socket says 172.21.15.99:38534) 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.697+0000 7fb979e00640 1 -- 172.21.15.99:0/765624366 learned_addr learned my addr 172.21.15.99:0/765624366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 -- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 msgr2=0x55c38120a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c38120a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 -- 172.21.15.99:0/765624366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 msgr2=0x55c381211a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c381211a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 -- 172.21.15.99:0/765624366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c380e703f0 con 0x55c381200200 2024-03-23T23:44:56.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb979e00640 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006010 tx=0x7fb96c0f42c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c843b1f089fd40 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb95fe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:56.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0bd050 con 0x55c381200200 2024-03-23T23:44:56.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb96c0b9dc0 con 0x55c381200200 2024-03-23T23:44:56.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.698+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9740178e0 con 0x55c381200200 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb96c004c80 con 0x55c381200200 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb983f9e740 10 client.25649 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c380f3a7d0 con 0x55c381200200 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9740178e0 con 0x55c381200200 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb96c0bd950 con 0x55c381200200 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb983f9e740 10 client.25649 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb983f9e740 10 client.25649 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.699+0000 7fb983f9e740 20 client.25649 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c381215f20 0x55c381218340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb983f9e740 4 client.25649 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c381169800 con 0x55c381215f20 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb973e00640 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c381215f20 0x55c381218340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb973e00640 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c381215f20 0x55c381218340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1527c07f752b564 server_cookie=23d0f37a46fe774d in_seq=0 out_seq=0 2024-03-23T23:44:56.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.700+0000 7fb95fe00640 10 client.25649 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.701+0000 7fb95fe00640 1 -- 172.21.15.99:0/765624366 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c381169800 con 0x55c381215f20 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.701+0000 7fb95fe00640 10 client.25649 handle_command_reply: tid=0 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 client.25649 shutdown 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 msgr2=0x55c38120a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006010 tx=0x7fb96c0f42c0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 shutdown_connections 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c381215f20 0x55c381218340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c381211a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c38120a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 --2- 172.21.15.99:0/765624366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c38120a4b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 >> 172.21.15.99:0/765624366 conn(0x55c3811fb5f0 msgr2=0x55c3811fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb95fe00640 0 client.25649 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:56.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 shutdown_connections 2024-03-23T23:44:56.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 1 -- 172.21.15.99:0/765624366 wait complete. 2024-03-23T23:44:56.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.702+0000 7fb983f9e740 20 client.25649 trim_cache size 0 max 16384 2024-03-23T23:44:56.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 Processor -- start 2024-03-23T23:44:56.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 -- start start 2024-03-23T23:44:56.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c3811ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c3811f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a6710 con 0x55c381201d90 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a8090 con 0x55c381200200 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.716+0000 7fb983f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a6d70 con 0x55c3812013d0 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb979e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c3811ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48550/0 (socket says 172.21.15.99:48550) 2024-03-23T23:44:56.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 -- 172.21.15.99:0/2664141154 learned_addr learned my addr 172.21.15.99:0/2664141154 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb97a800640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c3811f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 -- 172.21.15.99:0/2664141154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 msgr2=0x55c3811ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c3811ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 -- 172.21.15.99:0/2664141154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 msgr2=0x55c3811f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c3811f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb973e00640 1 -- 172.21.15.99:0/2664141154 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3812189c0 con 0x55c3812013d0 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb979e00640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c3811ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.717+0000 7fb97a800640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c3811f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.718+0000 7fb973e00640 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55c3811fc560 tx=0x7fb96401a960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=652eab73213b1846 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.718+0000 7fb96aa00640 1 -- 172.21.15.99:0/2664141154 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964005220 con 0x55c3812013d0 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.718+0000 7fb96aa00640 1 -- 172.21.15.99:0/2664141154 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9640053c0 con 0x55c3812013d0 2024-03-23T23:44:56.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.718+0000 7fb96aa00640 1 -- 172.21.15.99:0/2664141154 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964011820 con 0x55c3812013d0 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- 172.21.15.99:0/2664141154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 msgr2=0x55c3812119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55c3811fc560 tx=0x7fb96401a960 comp rx=0 tx=0).stop 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- 172.21.15.99:0/2664141154 shutdown_connections 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c381201d90 0x55c3811f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3812013d0 0x55c3812119c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 --2- 172.21.15.99:0/2664141154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381200200 0x55c3811ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- 172.21.15.99:0/2664141154 >> 172.21.15.99:0/2664141154 conn(0x55c3811fb5f0 msgr2=0x55c3811f18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- 172.21.15.99:0/2664141154 shutdown_connections 2024-03-23T23:44:56.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- 172.21.15.99:0/2664141154 wait complete. 2024-03-23T23:44:56.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.719+0000 7fb983f9e740 1 -- start start 2024-03-23T23:44:56.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 Processor -- start 2024-03-23T23:44:56.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c38120fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381210e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a7a30 con 0x55c3812013d0 2024-03-23T23:44:56.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a73d0 con 0x55c381201d90 2024-03-23T23:44:56.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1a48f0 con 0x55c381200200 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.720+0000 7fb983f9e740 10 client.0 _ll_register_callbacks cb 0x7fb96c0f4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48558/0 (socket says 172.21.15.99:48558) 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 -- 172.21.15.99:0/4273523239 learned_addr learned my addr 172.21.15.99:0/4273523239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb97a800640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381210e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb973e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c38120fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 -- 172.21.15.99:0/4273523239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 msgr2=0x55c381210e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381210e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 -- 172.21.15.99:0/4273523239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 msgr2=0x55c38120fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c38120fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 -- 172.21.15.99:0/4273523239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c381211410 con 0x55c381200200 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb979e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0bdd50 tx=0x7fb96c0f4800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4af2a6cbd5b8592 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:56.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.721+0000 7fb95fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:56.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.722+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0bd230 con 0x55c381200200 2024-03-23T23:44:56.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.722+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb96c0bcb40 con 0x55c381200200 2024-03-23T23:44:56.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.722+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9640053c0 con 0x55c381200200 2024-03-23T23:44:56.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb96c004c80 con 0x55c381200200 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 10 client.25687 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c381272010 con 0x55c381200200 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9640053c0 con 0x55c381200200 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb96c0b9440 con 0x55c381200200 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 10 client.25687 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 10 client.25687 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 20 client.25687 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c3812727c0 0x55c381274be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 4 client.25687 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.723+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c381163b50 con 0x55c3812727c0 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.724+0000 7fb973e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c3812727c0 0x55c381274be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:56.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.724+0000 7fb95fe00640 10 client.25687 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.724+0000 7fb973e00640 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c3812727c0 0x55c381274be0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d34fa8aed86e2f9 server_cookie=7b222f861638ee1 in_seq=0 out_seq=0 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb95fe00640 1 -- 172.21.15.99:0/4273523239 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c381163b50 con 0x55c3812727c0 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb95fe00640 10 client.25687 handle_command_reply: tid=0 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 client.25687 shutdown 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 msgr2=0x55c38120f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0bdd50 tx=0x7fb96c0f4800 comp rx=0 tx=0).stop 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 shutdown_connections 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c381201d90 0x55c381210e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55c3812727c0 0x55c381274be0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3812013d0 0x55c38120fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 --2- 172.21.15.99:0/4273523239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c381200200 0x55c38120f8d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 >> 172.21.15.99:0/4273523239 conn(0x55c3811fb5f0 msgr2=0x55c3811fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb95fe00640 0 client.25687 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 shutdown_connections 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.725+0000 7fb983f9e740 1 -- 172.21.15.99:0/4273523239 wait complete. 2024-03-23T23:44:56.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.726+0000 7fb983f9e740 20 client.25687 trim_cache size 0 max 16384 2024-03-23T23:44:56.728 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 msgr2=0x7fb9600a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 0x7fb9600a4e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb97401f000 tx=0x7fb974020000 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 msgr2=0x7fb97c199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000c00 tx=0x7fb96c005dd0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 shutdown_connections 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9600a29d0 0x7fb9600a4e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb97c107260 0x7fb97c1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb97c104930 0x7fb97c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 --2- 172.21.15.99:0/3920379972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb97c102000 0x7fb97c199b10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 >> 172.21.15.99:0/3920379972 conn(0x7fb97c0fba10 msgr2=0x7fb97c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 shutdown_connections 2024-03-23T23:44:56.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:56.727+0000 7fb97b200640 1 -- 172.21.15.99:0/3920379972 wait complete. 2024-03-23T23:44:56.750 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:56.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.026+0000 7fe7b1200640 1 -- 172.21.15.99:0/3277356093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac0ffdf0 msgr2=0x7fe7ac100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.026+0000 7fe7b1200640 1 --2- 172.21.15.99:0/3277356093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac0ffdf0 0x7fe7ac100210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe798009900 tx=0x7fe79802bc50 comp rx=0 tx=0).stop 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.026+0000 7fe7b1200640 1 -- 172.21.15.99:0/3277356093 shutdown_connections 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.027+0000 7fe7b1200640 1 --2- 172.21.15.99:0/3277356093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.027+0000 7fe7b1200640 1 --2- 172.21.15.99:0/3277356093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac100750 0x7fe7ac100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.027+0000 7fe7b1200640 1 --2- 172.21.15.99:0/3277356093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac0ffdf0 0x7fe7ac100210 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe798009900 tx=0x7fe79802bc50 comp rx=0 tx=0).stop 2024-03-23T23:44:57.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.027+0000 7fe7b1200640 1 -- 172.21.15.99:0/3277356093 >> 172.21.15.99:0/3277356093 conn(0x7fe7ac0fba10 msgr2=0x7fe7ac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.028+0000 7fe7b1200640 1 -- 172.21.15.99:0/3277356093 shutdown_connections 2024-03-23T23:44:57.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.028+0000 7fe7b1200640 1 -- 172.21.15.99:0/3277356093 wait complete. 2024-03-23T23:44:57.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.029+0000 7fe7b1200640 1 Processor -- start 2024-03-23T23:44:57.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.029+0000 7fe7b1200640 1 -- start start 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 0x7fe7ac19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 0x7fe7ac1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac102b90 con 0x7fe7ac1a3610 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac102a10 con 0x7fe7ac1010f0 2024-03-23T23:44:57.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7b1200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac102d10 con 0x7fe7ac100750 2024-03-23T23:44:57.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 0x7fe7ac19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 0x7fe7ac19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48570/0 (socket says 172.21.15.99:48570) 2024-03-23T23:44:57.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7ab400640 1 -- 172.21.15.99:0/4196568921 learned_addr learned my addr 172.21.15.99:0/4196568921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.030+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7abe00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 0x7fe7ac1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7aaa00640 1 -- 172.21.15.99:0/4196568921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 msgr2=0x7fe7ac19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 0x7fe7ac19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7aaa00640 1 -- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 msgr2=0x7fe7ac1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 0x7fe7ac1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7aaa00640 1 -- 172.21.15.99:0/4196568921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac1a6070 con 0x7fe7ac1010f0 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.031+0000 7fe7abe00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 0x7fe7ac1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.032+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac19e5b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00d6d0 tx=0x7fe79c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.032+0000 7fe7a3e00640 1 -- 172.21.15.99:0/4196568921 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c017070 con 0x7fe7ac1010f0 2024-03-23T23:44:57.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.032+0000 7fe7a3e00640 1 -- 172.21.15.99:0/4196568921 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79c009820 con 0x7fe7ac1010f0 2024-03-23T23:44:57.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.032+0000 7fe7a3e00640 1 -- 172.21.15.99:0/4196568921 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c004cd0 con 0x7fe7ac1010f0 2024-03-23T23:44:57.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.032+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac1a6360 con 0x7fe7ac1010f0 2024-03-23T23:44:57.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.033+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac1a67a0 con 0x7fe7ac1010f0 2024-03-23T23:44:57.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.036+0000 7fe7a3e00640 1 -- 172.21.15.99:0/4196568921 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe79c0099d0 con 0x7fe7ac1010f0 2024-03-23T23:44:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.037+0000 7fe7b5404740 1 Processor -- start 2024-03-23T23:44:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.038+0000 7fe7b5404740 1 -- start start 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.038+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa87392e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.038+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa8739ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7b5404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa872f72c0 con 0x55fa87394580 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7b5404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8729fc90 con 0x55fa873929f0 2024-03-23T23:44:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7b5404740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa872f3360 con 0x55fa87393bc0 2024-03-23T23:44:57.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7a3e00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 0x7fe7900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.039+0000 7fe7a3e00640 1 -- 172.21.15.99:0/4196568921 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe79c0ce550 con 0x7fe7ac1010f0 2024-03-23T23:44:57.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa8739ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa87392e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48586/0 (socket says 172.21.15.99:48586) 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 -- 172.21.15.99:0/2139450255 learned_addr learned my addr 172.21.15.99:0/2139450255 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 -- 172.21.15.99:0/2139450255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 msgr2=0x55fa87392e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa87392e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7ab400640 1 -- 172.21.15.99:0/2139450255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 msgr2=0x55fa8739ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.040+0000 7fe7abe00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 0x7fe7900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.041+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa8739ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.041+0000 7fe7ab400640 1 -- 172.21.15.99:0/2139450255 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa872c4080 con 0x55fa87393bc0 2024-03-23T23:44:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.041+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa8739ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:57.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.041+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe798005ab0 tx=0x7fe7980095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e340b6c9d22634 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.042+0000 7fe7a0c00640 1 -- 172.21.15.99:0/2139450255 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798045020 con 0x55fa87393bc0 2024-03-23T23:44:57.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.042+0000 7fe7a0c00640 1 -- 172.21.15.99:0/2139450255 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79802c9e0 con 0x55fa87393bc0 2024-03-23T23:44:57.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.042+0000 7fe7abe00640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 0x7fe7900a4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40099e0 tx=0x7fe7a4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7a0c00640 1 -- 172.21.15.99:0/2139450255 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980046c0 con 0x55fa87393bc0 2024-03-23T23:44:57.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 -- 172.21.15.99:0/2139450255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 msgr2=0x55fa87394040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe798005ab0 tx=0x7fe7980095e0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 -- 172.21.15.99:0/2139450255 shutdown_connections 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa8739ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa87394040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2139450255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa87392e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 -- 172.21.15.99:0/2139450255 >> 172.21.15.99:0/2139450255 conn(0x55fa8738dde0 msgr2=0x55fa8738e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 -- 172.21.15.99:0/2139450255 shutdown_connections 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.043+0000 7fe7b5404740 1 -- 172.21.15.99:0/2139450255 wait complete. 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.044+0000 7fe7b5404740 1 -- start start 2024-03-23T23:44:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.044+0000 7fe7b5404740 1 Processor -- start 2024-03-23T23:44:57.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa8739d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 0x55fa873a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 0x55fa873a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 0x55fa873a4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38620/0 (socket says 172.21.15.99:38620) 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa8739d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.045+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48090/0 (socket says 172.21.15.99:48090) 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa8739d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48590/0 (socket says 172.21.15.99:48590) 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8739b580 con 0x55fa873929f0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7abe00640 1 -- 172.21.15.99:0/3174774665 learned_addr learned my addr 172.21.15.99:0/3174774665 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa872b30d0 con 0x55fa87394580 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8739b700 con 0x55fa87393bc0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7b5404740 10 client.0 _ll_register_callbacks cb 0x55fa872b7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7ab400640 1 -- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 msgr2=0x55fa8739d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7ab400640 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa8739d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7ab400640 1 -- 172.21.15.99:0/3174774665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 msgr2=0x55fa873a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7ab400640 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 0x55fa873a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.046+0000 7fe7ab400640 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa8700b3f0 con 0x55fa873929f0 2024-03-23T23:44:57.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.047+0000 7fe7ab400640 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe79802c260 tx=0x7fe79802cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267cddef85aa6082 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.047+0000 7fe795600640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.047+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798045030 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.047+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe798041a50 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.048+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe79802c9e0 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.048+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe798041c50 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.048+0000 7fe7b5404740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.048+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa870d57d0 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79802c9e0 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe79804c440 con 0x55fa873929f0 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 10 client.15531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 20 client.15531 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa873a8270 0x55fa873aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 4 client.15531 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa87304800 con 0x55fa873a8270 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.049+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa873a8270 0x55fa873aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.050+0000 7fe795600640 10 client.15531 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:57.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.050+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa873a8270 0x55fa873aa660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7fc5ce6227cbdb48 server_cookie=ff827bdda11e4066 in_seq=0 out_seq=0 2024-03-23T23:44:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe795600640 1 -- 172.21.15.99:0/3174774665 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fa87304800 con 0x55fa873a8270 2024-03-23T23:44:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe795600640 10 client.15531 handle_command_reply: tid=0 2024-03-23T23:44:57.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 client.15531 shutdown 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 msgr2=0x55fa8739cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe79802c260 tx=0x7fe79802cdc0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 shutdown_connections 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa873a8270 0x55fa873aa660 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa87394580 0x55fa873a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa8739d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 --2- 172.21.15.99:0/3174774665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa873929f0 0x55fa8739cb90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.051+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 >> 172.21.15.99:0/3174774665 conn(0x55fa8738dde0 msgr2=0x55fa8738d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.052+0000 7fe795600640 0 client.15531 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.052+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 shutdown_connections 2024-03-23T23:44:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.052+0000 7fe7b5404740 1 -- 172.21.15.99:0/3174774665 wait complete. 2024-03-23T23:44:57.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.052+0000 7fe7b5404740 20 client.15531 trim_cache size 0 max 16384 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 Processor -- start 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 -- start start 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87393bc0 0x55fa873a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87394580 0x55fa873aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1af370 con 0x55fa87393bc0 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1b0cf0 con 0x55fa873929f0 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.078+0000 7fe7b5404740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1af9d0 con 0x55fa87394580 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87393bc0 0x55fa873a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38622/0 (socket says 172.21.15.99:38622) 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 -- 172.21.15.99:0/2384397794 learned_addr learned my addr 172.21.15.99:0/2384397794 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7abe00640 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87394580 0x55fa873aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 -- 172.21.15.99:0/2384397794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87394580 msgr2=0x55fa873aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87394580 0x55fa873aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 -- 172.21.15.99:0/2384397794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87393bc0 msgr2=0x55fa873a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87393bc0 0x55fa873a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.079+0000 7fe7ab400640 1 -- 172.21.15.99:0/2384397794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa873898b0 con 0x55fa873929f0 2024-03-23T23:44:57.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.080+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe79802c190 tx=0x7fe798041480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6491eaf0bd68aa6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.080+0000 7fe797e00640 1 -- 172.21.15.99:0/2384397794 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798045070 con 0x55fa873929f0 2024-03-23T23:44:57.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.080+0000 7fe797e00640 1 -- 172.21.15.99:0/2384397794 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79802c9e0 con 0x55fa873929f0 2024-03-23T23:44:57.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.080+0000 7fe797e00640 1 -- 172.21.15.99:0/2384397794 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798045070 con 0x55fa873929f0 2024-03-23T23:44:57.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 -- 172.21.15.99:0/2384397794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 msgr2=0x55fa8738c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe79802c190 tx=0x7fe798041480 comp rx=0 tx=0).stop 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 -- 172.21.15.99:0/2384397794 shutdown_connections 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87394580 0x55fa873aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87393bc0 0x55fa873a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2384397794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa8738c8d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 -- 172.21.15.99:0/2384397794 >> 172.21.15.99:0/2384397794 conn(0x55fa8738dde0 msgr2=0x55fa873918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 -- 172.21.15.99:0/2384397794 shutdown_connections 2024-03-23T23:44:57.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.082+0000 7fe7b5404740 1 -- 172.21.15.99:0/2384397794 wait complete. 2024-03-23T23:44:57.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.083+0000 7fe7b5404740 1 -- start start 2024-03-23T23:44:57.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.083+0000 7fe7b5404740 1 Processor -- start 2024-03-23T23:44:57.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.084+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa873a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.084+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.084+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.084+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa873a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.084+0000 7fe7aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48606/0 (socket says 172.21.15.99:48606) 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa873a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38634/0 (socket says 172.21.15.99:38634) 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7b5404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa873a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7b5404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1b0690 con 0x55fa87394580 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7b5404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1b0030 con 0x55fa873929f0 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7b5404740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1ad530 con 0x55fa87393bc0 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7b5404740 10 client.0 _ll_register_callbacks cb 0x7fe79c09acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7aaa00640 1 -- 172.21.15.99:0/2383020737 learned_addr learned my addr 172.21.15.99:0/2383020737 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7abe00640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa873a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7aaa00640 1 -- 172.21.15.99:0/2383020737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 msgr2=0x55fa873a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.085+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa873a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe7aaa00640 1 -- 172.21.15.99:0/2383020737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 msgr2=0x55fa873a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa873a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe7aaa00640 1 -- 172.21.15.99:0/2383020737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa873a3840 con 0x55fa87393bc0 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe7abe00640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa873a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe7aaa00640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe79c09ac20 tx=0x7fe79c0cc0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05436c99063b4ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe795600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.086+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c020040 con 0x55fa87393bc0 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.087+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe79c0ce2b0 con 0x55fa87393bc0 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.087+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe79802c9e0 con 0x55fa87393bc0 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.087+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe79c09a370 con 0x55fa87393bc0 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.087+0000 7fe7b5404740 10 client.25705 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.087+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa87403eb0 con 0x55fa87393bc0 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79802c9e0 con 0x55fa87393bc0 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe79c0b1030 con 0x55fa87393bc0 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 10 client.25705 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 20 client.25705 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa87404780 0x55fa87406c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 4 client.25705 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa872feb50 con 0x55fa87404780 2024-03-23T23:44:57.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.088+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa87404780 0x55fa87406c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.089+0000 7fe7ab400640 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa87404780 0x55fa87406c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfc9c9f432af5018 server_cookie=2188e0583c5e8722 in_seq=0 out_seq=0 2024-03-23T23:44:57.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.089+0000 7fe795600640 10 client.25705 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:57.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.089+0000 7fe795600640 1 -- 172.21.15.99:0/2383020737 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa872feb50 con 0x55fa87404780 2024-03-23T23:44:57.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe795600640 10 client.25705 handle_command_reply: tid=0 2024-03-23T23:44:57.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 client.25705 shutdown 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 msgr2=0x55fa873a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe79c09ac20 tx=0x7fe79c0cc0e0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 shutdown_connections 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55fa87404780 0x55fa87406c20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa87394580 0x55fa873a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa87393bc0 0x55fa873a2240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 --2- 172.21.15.99:0/2383020737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa873929f0 0x55fa873a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 >> 172.21.15.99:0/2383020737 conn(0x55fa8738dde0 msgr2=0x55fa87396df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.090+0000 7fe795600640 0 client.25705 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.091+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 shutdown_connections 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.091+0000 7fe7b5404740 1 -- 172.21.15.99:0/2383020737 wait complete. 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.091+0000 7fe7b5404740 20 client.25705 trim_cache size 0 max 16384 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 msgr2=0x7fe7900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 0x7fe7900a4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40099e0 tx=0x7fe7a4004ab0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 msgr2=0x7fe7ac19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac19e5b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00d6d0 tx=0x7fe79c00db90 comp rx=0 tx=0).stop 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 shutdown_connections 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe7900a2880 0x7fe7900a4d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7ac1a3610 0x7fe7ac1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7ac1010f0 0x7fe7ac19e5b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.093+0000 7fe7b1200640 1 --2- 172.21.15.99:0/4196568921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7ac100750 0x7fe7ac19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.094+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 >> 172.21.15.99:0/4196568921 conn(0x7fe7ac0fba10 msgr2=0x7fe7ac0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.095+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 shutdown_connections 2024-03-23T23:44:57.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.095+0000 7fe7b1200640 1 -- 172.21.15.99:0/4196568921 wait complete. 2024-03-23T23:44:57.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:56 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2458257138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:57.116 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:57.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 -- 172.21.15.99:0/3518448278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd00772a0 msgr2=0x7f9cd0077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/3518448278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd00772a0 0x7f9cd0077700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009860 tx=0x7f9cbc02bbb0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 -- 172.21.15.99:0/3518448278 shutdown_connections 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/3518448278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd0077c40 0x7f9cd00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/3518448278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd00772a0 0x7f9cd0077700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/3518448278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd0076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 -- 172.21.15.99:0/3518448278 >> 172.21.15.99:0/3518448278 conn(0x7f9cd00fee90 msgr2=0x7f9cd01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.296+0000 7f9cd5c00640 1 -- 172.21.15.99:0/3518448278 shutdown_connections 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.297+0000 7f9cd5c00640 1 -- 172.21.15.99:0/3518448278 wait complete. 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.298+0000 7f9cd5c00640 1 Processor -- start 2024-03-23T23:44:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.298+0000 7f9cd5c00640 1 -- start start 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.299+0000 7f9cd5c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd010e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.299+0000 7f9cd5c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd00772a0 0x7f9cd010e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9cd5c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9cd5c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01139d0 con 0x7f9cd00772a0 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9cd5c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0113850 con 0x7f9cd0076060 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9cd5c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0113b50 con 0x7f9cd0077c40 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd010e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48618/0 (socket says 172.21.15.99:48618) 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 -- 172.21.15.99:0/2268076472 learned_addr learned my addr 172.21.15.99:0/2268076472 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 -- 172.21.15.99:0/2268076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 msgr2=0x7f9cd010e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd010e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.300+0000 7f9ccf400640 1 -- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd00772a0 msgr2=0x7f9cd010e7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T23:44:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9cce000640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd00772a0 0x7f9cd010e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9ccf400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd00772a0 0x7f9cd010e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9ccf400640 1 -- 172.21.15.99:0/2268076472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd01118f0 con 0x7f9cd0077c40 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9ccf400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800eaa0 tx=0x7f9cc800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9cc7400640 1 -- 172.21.15.99:0/2268076472 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800cdb0 con 0x7f9cd0077c40 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.301+0000 7f9cc7400640 1 -- 172.21.15.99:0/2268076472 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc8005020 con 0x7f9cd0077c40 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.302+0000 7f9cc7400640 1 -- 172.21.15.99:0/2268076472 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80116b0 con 0x7f9cd0077c40 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.302+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd01ae6a0 con 0x7f9cd0077c40 2024-03-23T23:44:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.302+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd01aeb70 con 0x7f9cd0077c40 2024-03-23T23:44:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.309+0000 7f9cc7400640 1 -- 172.21.15.99:0/2268076472 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9cc8004b20 con 0x7f9cd0077c40 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.310+0000 7f9cd89cb740 1 Processor -- start 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 -- start start 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abdc43b0 con 0x5580abe011e0 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abd66830 con 0x5580abe06d40 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.311+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abd0f5d0 con 0x5580abe01bd0 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.313+0000 7f9cce000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.313+0000 7f9cce000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe01600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48136/0 (socket says 172.21.15.99:48136) 2024-03-23T23:44:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.313+0000 7f9cce000640 1 -- 172.21.15.99:0/2065708196 learned_addr learned my addr 172.21.15.99:0/2065708196 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.313+0000 7f9cce000640 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.313+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9cc7400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 0x7f9cb40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9cc7400640 1 -- 172.21.15.99:0/2268076472 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cc80ced10 con 0x7f9cd0077c40 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9ccea00640 1 -- 172.21.15.99:0/2065708196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 msgr2=0x5580abe091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9ccea00640 1 -- 172.21.15.99:0/2065708196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 msgr2=0x5580abe01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe01600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.315+0000 7f9ccea00640 1 -- 172.21.15.99:0/2065708196 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580abd272c0 con 0x5580abe01bd0 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.316+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe066d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009b70 tx=0x7f9cc0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c707375c39720ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.317+0000 7f9cbbe00640 1 -- 172.21.15.99:0/2065708196 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0009e60 con 0x5580abe01bd0 2024-03-23T23:44:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.317+0000 7f9cbbe00640 1 -- 172.21.15.99:0/2065708196 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc0003610 con 0x5580abe01bd0 2024-03-23T23:44:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.317+0000 7f9cbbe00640 1 -- 172.21.15.99:0/2065708196 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00126b0 con 0x5580abe01bd0 2024-03-23T23:44:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.317+0000 7f9ccf400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 0x7f9cb40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.318+0000 7f9ccf400640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 0x7f9cb40a4e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004af0 tx=0x7f9cc8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2065708196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 msgr2=0x5580abe066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe066d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009b70 tx=0x7f9cc0010740 comp rx=0 tx=0).stop 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2065708196 shutdown_connections 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe066d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2065708196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2065708196 >> 172.21.15.99:0/2065708196 conn(0x5580abdfc750 msgr2=0x5580abdfcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2065708196 shutdown_connections 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.319+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2065708196 wait complete. 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.320+0000 7f9cd89cb740 1 -- start start 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.320+0000 7f9cd89cb740 1 Processor -- start 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe12890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abd22a20 con 0x5580abe011e0 2024-03-23T23:44:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abd628d0 con 0x5580abe06d40 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580abe09cf0 con 0x5580abe01bd0 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cd89cb740 10 client.0 _ll_register_callbacks cb 0x5580abd5f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48630/0 (socket says 172.21.15.99:48630) 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9ccea00640 1 -- 172.21.15.99:0/3051630831 learned_addr learned my addr 172.21.15.99:0/3051630831 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.321+0000 7f9cce000640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.322+0000 7f9ccea00640 1 -- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 msgr2=0x5580abe12890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.322+0000 7f9ccea00640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe12890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.322+0000 7f9ccea00640 1 -- 172.21.15.99:0/3051630831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 msgr2=0x5580abe0b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.322+0000 7f9ccea00640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.322+0000 7f9ccea00640 1 -- 172.21.15.99:0/3051630831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580aba7b210 con 0x5580abe01bd0 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9ccea00640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010c40 tx=0x7f9cc0003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f0ab06c91971fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc002a060 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cc00039c0 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc0003610 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc0009770 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cd89cb740 10 client.25717 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.323+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580abb47620 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.324+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc0003610 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.325+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9cc000f040 con 0x5580abe01bd0 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.325+0000 7f9cd89cb740 10 client.25717 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.325+0000 7f9cd89cb740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.325+0000 7f9cd89cb740 20 client.25717 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.325+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe16e30 0x5580abe19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.326+0000 7f9cd89cb740 4 client.25717 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.326+0000 7f9cce000640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe16e30 0x5580abe19220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.326+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580abd73d70 con 0x5580abe16e30 2024-03-23T23:44:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.327+0000 7f9cce000640 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe16e30 0x5580abe19220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976f9eb5647d27d8 server_cookie=b9600fb5bdd177f3 in_seq=0 out_seq=0 2024-03-23T23:44:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.328+0000 7f9cb8c00640 10 client.25717 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.328+0000 7f9cb8c00640 1 -- 172.21.15.99:0/3051630831 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5580abd73d70 con 0x5580abe16e30 2024-03-23T23:44:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.328+0000 7f9cb8c00640 10 client.25717 handle_command_reply: tid=0 2024-03-23T23:44:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.328+0000 7f9cd89cb740 1 client.25717 shutdown 2024-03-23T23:44:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 msgr2=0x5580abe0b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010c40 tx=0x7f9cc0003db0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 shutdown_connections 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe16e30 0x5580abe19220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe06d40 0x5580abe12890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe01bd0 0x5580abe0b850 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/3051630831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.329+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 >> 172.21.15.99:0/3051630831 conn(0x5580abdfc750 msgr2=0x5580abdfb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.330+0000 7f9cb8c00640 0 client.25717 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.330+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 shutdown_connections 2024-03-23T23:44:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.330+0000 7f9cd89cb740 1 -- 172.21.15.99:0/3051630831 wait complete. 2024-03-23T23:44:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.330+0000 7f9cd89cb740 20 client.25717 trim_cache size 0 max 16384 2024-03-23T23:44:57.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.359+0000 7f9cd89cb740 1 Processor -- start 2024-03-23T23:44:57.359 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.359+0000 7f9cd89cb740 1 -- start start 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 0x5580abdfb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b7710 con 0x5580abe12850 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b9090 con 0x5580abe011e0 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b7d70 con 0x5580abe06d40 2024-03-23T23:44:57.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48636/0 (socket says 172.21.15.99:48636) 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 -- 172.21.15.99:0/2375116654 learned_addr learned my addr 172.21.15.99:0/2375116654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 -- 172.21.15.99:0/2375116654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 msgr2=0x5580abe0b310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.360+0000 7f9ccea00640 1 -- 172.21.15.99:0/2375116654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 msgr2=0x5580abdfb0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.361+0000 7f9cce000640 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 0x5580abdfb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.361+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 0x5580abdfb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.361+0000 7f9ccea00640 1 -- 172.21.15.99:0/2375116654 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580abdf73a0 con 0x5580abe06d40 2024-03-23T23:44:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.361+0000 7f9cce000640 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 0x5580abdfb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.362+0000 7f9ccea00640 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010780 tx=0x7f9cc0005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1473702dfbc64ded server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.362+0000 7f9cbb400640 1 -- 172.21.15.99:0/2375116654 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc002a070 con 0x5580abe06d40 2024-03-23T23:44:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.363+0000 7f9cbb400640 1 -- 172.21.15.99:0/2375116654 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc001aca0 con 0x5580abe06d40 2024-03-23T23:44:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.363+0000 7f9cbb400640 1 -- 172.21.15.99:0/2375116654 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003830 con 0x5580abe06d40 2024-03-23T23:44:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2375116654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 msgr2=0x5580abe0b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010780 tx=0x7f9cc0005970 comp rx=0 tx=0).stop 2024-03-23T23:44:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2375116654 shutdown_connections 2024-03-23T23:44:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe12850 0x5580abdfb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe06d40 0x5580abe0b850 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/2375116654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.364+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2375116654 >> 172.21.15.99:0/2375116654 conn(0x5580abdfc750 msgr2=0x5580abe02ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.365+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2375116654 shutdown_connections 2024-03-23T23:44:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.365+0000 7f9cd89cb740 1 -- 172.21.15.99:0/2375116654 wait complete. 2024-03-23T23:44:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.366+0000 7f9cd89cb740 1 -- start start 2024-03-23T23:44:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.366+0000 7f9cd89cb740 1 Processor -- start 2024-03-23T23:44:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.367+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.367+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 0x5580abe103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cce000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cce000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38680/0 (socket says 172.21.15.99:38680) 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cd89cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe12850 0x5580abe113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b8a30 con 0x5580abe06d40 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b83d0 con 0x5580abe011e0 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cd89cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01b5980 con 0x5580abe12850 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.368+0000 7f9cd89cb740 10 client.0 _ll_register_callbacks cb 0x7f9cc00099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.369+0000 7f9cce000640 1 -- 172.21.15.99:0/611604878 learned_addr learned my addr 172.21.15.99:0/611604878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.369+0000 7f9cce000640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe12850 0x5580abe113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.369+0000 7f9cce000640 1 -- 172.21.15.99:0/611604878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe12850 msgr2=0x5580abe113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.369+0000 7f9cce000640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe12850 0x5580abe113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.369+0000 7f9cce000640 1 -- 172.21.15.99:0/611604878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 msgr2=0x5580abe103b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.370+0000 7f9ccea00640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 0x5580abe103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.370+0000 7f9cce000640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 0x5580abe103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.370+0000 7f9cce000640 1 -- 172.21.15.99:0/611604878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580abe119b0 con 0x5580abe011e0 2024-03-23T23:44:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.370+0000 7f9ccea00640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 0x5580abe103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.371+0000 7f9cce000640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02c140 tx=0x7f9cbc02c0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31dc1151867a6c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.371+0000 7f9cb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.371+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc039040 con 0x5580abe011e0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.371+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cbc0490e0 con 0x5580abe011e0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.371+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc001aca0 con 0x5580abe011e0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.372+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc004030 con 0x5580abe011e0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.372+0000 7f9cd89cb740 10 client.25682 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.372+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580abe72b40 con 0x5580abe011e0 2024-03-23T23:44:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.372+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc001aca0 con 0x5580abe011e0 2024-03-23T23:44:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.373+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f9cbc0035f0 con 0x5580abe011e0 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 10 client.25682 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 20 client.25682 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe73410 0x5580abe75830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 4 client.25682 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:44:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580abd6e0c0 con 0x5580abe73410 2024-03-23T23:44:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.374+0000 7f9ccea00640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe73410 0x5580abe75830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.375+0000 7f9ccea00640 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe73410 0x5580abe75830 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95eac230f73d9a7c server_cookie=b7065a1aa7f4a5bc in_seq=0 out_seq=0 2024-03-23T23:44:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.375+0000 7f9cb8c00640 10 client.25682 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.379+0000 7f9cb8c00640 1 -- 172.21.15.99:0/611604878 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580abd6e0c0 con 0x5580abe73410 2024-03-23T23:44:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.379+0000 7f9cb8c00640 10 client.25682 handle_command_reply: tid=0 2024-03-23T23:44:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.379+0000 7f9cd89cb740 1 client.25682 shutdown 2024-03-23T23:44:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:44:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:44:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:44:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.379+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 msgr2=0x5580abe0fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.379+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02c140 tx=0x7f9cbc02c0d0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 shutdown_connections 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5580abe73410 0x5580abe75830 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580abe12850 0x5580abe113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580abe06d40 0x5580abe103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 --2- 172.21.15.99:0/611604878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580abe011e0 0x5580abe0fe70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cb8c00640 0 client.25682 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 >> 172.21.15.99:0/611604878 conn(0x5580abdfc750 msgr2=0x5580abe02ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 shutdown_connections 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 1 -- 172.21.15.99:0/611604878 wait complete. 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.381+0000 7f9cd89cb740 20 client.25682 trim_cache size 0 max 16384 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.382+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 msgr2=0x7f9cb40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.382+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 0x7f9cb40a4e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004af0 tx=0x7f9cc8004a40 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.382+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 msgr2=0x7f9cd01ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.382+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800eaa0 tx=0x7f9cc800ef60 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 shutdown_connections 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f9cb40a2970 0x7f9cb40a4e30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cd0077c40 0x7f9cd01ae0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd00772a0 0x7f9cd010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 --2- 172.21.15.99:0/2268076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd0076060 0x7f9cd010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.383+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 >> 172.21.15.99:0/2268076472 conn(0x7f9cd00fee90 msgr2=0x7f9cd0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.384+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 shutdown_connections 2024-03-23T23:44:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:44:57.384+0000 7f9cd5c00640 1 -- 172.21.15.99:0/2268076472 wait complete. 2024-03-23T23:44:57.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:56 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2458257138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:57.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:56 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2458257138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:44:57.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 -- 172.21.15.99:0/3239013136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc40ffdf0 msgr2=0x7fbfc4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3239013136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc40ffdf0 0x7fbfc4102220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009900 tx=0x7fbfb402bc30 comp rx=0 tx=0).stop 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 -- 172.21.15.99:0/3239013136 shutdown_connections 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3239013136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc4105110 0x7fbfc4105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3239013136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4102760 0x7fbfc4104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3239013136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc40ffdf0 0x7fbfc4102220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.436+0000 7fbfc9000640 1 -- 172.21.15.99:0/3239013136 >> 172.21.15.99:0/3239013136 conn(0x7fbfc40fba10 msgr2=0x7fbfc40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.437+0000 7fbfc9000640 1 -- 172.21.15.99:0/3239013136 shutdown_connections 2024-03-23T23:44:57.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.437+0000 7fbfc9000640 1 -- 172.21.15.99:0/3239013136 wait complete. 2024-03-23T23:44:57.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.438+0000 7fbfc9000640 1 Processor -- start 2024-03-23T23:44:57.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 -- start start 2024-03-23T23:44:57.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ffdf0 0x7fbfc406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 0x7fbfc40704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 0x7fbfc41adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4113820 con 0x7fbfc40ffdf0 2024-03-23T23:44:57.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41136a0 con 0x7fbfc4105110 2024-03-23T23:44:57.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.439+0000 7fbfc9000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41139a0 con 0x7fbfc4102760 2024-03-23T23:44:57.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 0x7fbfc41adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 0x7fbfc41adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38698/0 (socket says 172.21.15.99:38698) 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc3e00640 1 -- 172.21.15.99:0/3887256875 learned_addr learned my addr 172.21.15.99:0/3887256875 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 0x7fbfc40704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 -- 172.21.15.99:0/3887256875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 msgr2=0x7fbfc41adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 0x7fbfc41adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 -- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ffdf0 msgr2=0x7fbfc406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ffdf0 0x7fbfc406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.440+0000 7fbfc2a00640 1 -- 172.21.15.99:0/3887256875 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4073660 con 0x7fbfc4102760 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.441+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 0x7fbfc40704e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e9f0 tx=0x7fbfb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.441+0000 7fbfbbe00640 1 -- 172.21.15.99:0/3887256875 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000ccc0 con 0x7fbfc4102760 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.441+0000 7fbfbbe00640 1 -- 172.21.15.99:0/3887256875 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfb0010910 con 0x7fbfc4102760 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.441+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc41ae530 con 0x7fbfc4102760 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.441+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc41aeaa0 con 0x7fbfc4102760 2024-03-23T23:44:57.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.442+0000 7fbfbbe00640 1 -- 172.21.15.99:0/3887256875 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0018920 con 0x7fbfc4102760 2024-03-23T23:44:57.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.446+0000 7fbfcd42f740 1 Processor -- start 2024-03-23T23:44:57.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.447+0000 7fbfbbe00640 1 -- 172.21.15.99:0/3887256875 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbfb0018ac0 con 0x7fbfc4102760 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.447+0000 7fbfcd42f740 1 -- start start 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd54040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd54580 0x55ea4cd5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4ccb72c0 con 0x55ea4cd53bc0 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc5fc90 con 0x55ea4cd529f0 2024-03-23T23:44:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4ccb3360 con 0x55ea4cd54580 2024-03-23T23:44:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38712/0 (socket says 172.21.15.99:38712) 2024-03-23T23:44:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc3e00640 1 -- 172.21.15.99:0/4023541615 learned_addr learned my addr 172.21.15.99:0/4023541615 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd54580 0x55ea4cd5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc3400640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd54040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.448+0000 7fbfc3e00640 1 -- 172.21.15.99:0/4023541615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd54580 msgr2=0x55ea4cd5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd54580 0x55ea4cd5ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3e00640 1 -- 172.21.15.99:0/4023541615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 msgr2=0x55ea4cd54040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd54040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3e00640 1 -- 172.21.15.99:0/4023541615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea4cc84080 con 0x55ea4cd529f0 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3400640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd54040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007fc0 tx=0x7fbfbc00f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e638ece2adb01f42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.449+0000 7fbfbbe00640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 0x7fbfa80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.450+0000 7fbfbbe00640 1 -- 172.21.15.99:0/3887256875 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfb00cedf0 con 0x7fbfc4102760 2024-03-23T23:44:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.450+0000 7fbfb8c00640 1 -- 172.21.15.99:0/4023541615 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004d20 con 0x55ea4cd529f0 2024-03-23T23:44:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.450+0000 7fbfb8c00640 1 -- 172.21.15.99:0/4023541615 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc010be0 con 0x55ea4cd529f0 2024-03-23T23:44:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.450+0000 7fbfb8c00640 1 -- 172.21.15.99:0/4023541615 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc005810 con 0x55ea4cd529f0 2024-03-23T23:44:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfc3400640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 0x7fbfa80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 -- 172.21.15.99:0/4023541615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 msgr2=0x55ea4cd52e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007fc0 tx=0x7fbfbc00f850 comp rx=0 tx=0).stop 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 -- 172.21.15.99:0/4023541615 shutdown_connections 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd54580 0x55ea4cd5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd54040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/4023541615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd529f0 0x55ea4cd52e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.451+0000 7fbfcd42f740 1 -- 172.21.15.99:0/4023541615 >> 172.21.15.99:0/4023541615 conn(0x55ea4cd4dde0 msgr2=0x55ea4cd4e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.452+0000 7fbfcd42f740 1 -- 172.21.15.99:0/4023541615 shutdown_connections 2024-03-23T23:44:57.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.452+0000 7fbfcd42f740 1 -- 172.21.15.99:0/4023541615 wait complete. 2024-03-23T23:44:57.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.453+0000 7fbfc3400640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 0x7fbfa80a4d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ea4cd4ed50 tx=0x7fbfb402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.453+0000 7fbfcd42f740 1 -- start start 2024-03-23T23:44:57.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.453+0000 7fbfcd42f740 1 Processor -- start 2024-03-23T23:44:57.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cd5b590 con 0x55ea4cd54580 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc730d0 con 0x55ea4cd53bc0 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cd5b710 con 0x55ea4cd529f0 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfcd42f740 10 client.0 _ll_register_callbacks cb 0x55ea4cc77e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48684/0 (socket says 172.21.15.99:48684) 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfc3400640 1 -- 172.21.15.99:0/2791590175 learned_addr learned my addr 172.21.15.99:0/2791590175 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.454+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3400640 1 -- 172.21.15.99:0/2791590175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 msgr2=0x55ea4cd5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3400640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3400640 1 -- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 msgr2=0x55ea4cd64270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3400640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.455+0000 7fbfc3400640 1 -- 172.21.15.99:0/2791590175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea4c9cb3f0 con 0x55ea4cd529f0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfc3400640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4035000 tx=0x7fbfb4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb09fe402e2a135 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfad600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4046040 con 0x55ea4cd529f0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfb40055c0 con 0x55ea4cd529f0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfbc010be0 con 0x55ea4cd529f0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb4004380 con 0x55ea4cd529f0 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfcd42f740 10 client.25735 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.456+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea4ca957d0 con 0x55ea4cd529f0 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc010be0 con 0x55ea4cd529f0 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbfb4003920 con 0x55ea4cd529f0 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfcd42f740 10 client.25735 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfcd42f740 10 client.25735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfcd42f740 20 client.25735 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.457+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cd68760 0x55ea4cd6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.458+0000 7fbfcd42f740 4 client.25735 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:57.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.458+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea4ccc4800 con 0x55ea4cd68760 2024-03-23T23:44:57.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.458+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cd68760 0x55ea4cd6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.459+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cd68760 0x55ea4cd6abe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ae74ff564c87d2f server_cookie=bda06e1268b6188c in_seq=0 out_seq=0 2024-03-23T23:44:57.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.459+0000 7fbfad600640 10 client.25735 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:57.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.460+0000 7fbfad600640 1 -- 172.21.15.99:0/2791590175 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea4ccc4800 con 0x55ea4cd68760 2024-03-23T23:44:57.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.461+0000 7fbfad600640 10 client.25735 handle_command_reply: tid=0 2024-03-23T23:44:57.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.461+0000 7fbfcd42f740 1 client.25735 shutdown 2024-03-23T23:44:57.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.461+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 msgr2=0x55ea4cd5ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.461+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4035000 tx=0x7fbfb4005fd0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 shutdown_connections 2024-03-23T23:44:57.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cd68760 0x55ea4cd6abe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/2791590175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd529f0 0x55ea4cd5ccf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 >> 172.21.15.99:0/2791590175 conn(0x55ea4cd4dde0 msgr2=0x55ea4cd4d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfad600640 0 client.25735 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 shutdown_connections 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 1 -- 172.21.15.99:0/2791590175 wait complete. 2024-03-23T23:44:57.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.462+0000 7fbfcd42f740 20 client.25735 trim_cache size 0 max 16384 2024-03-23T23:44:57.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.488+0000 7fbfcd42f740 1 Processor -- start 2024-03-23T23:44:57.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.488+0000 7fbfcd42f740 1 -- start start 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400e3a0 con 0x55ea4cd54580 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400fd20 con 0x55ea4cd53bc0 2024-03-23T23:44:57.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400ea00 con 0x55ea4cd56c60 2024-03-23T23:44:57.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.489+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.490+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.490+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38734/0 (socket says 172.21.15.99:38734) 2024-03-23T23:44:57.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.490+0000 7fbfc3400640 1 -- 172.21.15.99:0/1181331453 learned_addr learned my addr 172.21.15.99:0/1181331453 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.490+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1181331453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 msgr2=0x55ea4cd51750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1181331453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 msgr2=0x55ea4cd64200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1181331453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea4cd68520 con 0x55ea4cd56c60 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3400640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:57.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc2a00640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:57.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.491+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd57110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc000f80 tx=0x7fbfbc007ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760f6d3b855cc70b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.492+0000 7fbfafe00640 1 -- 172.21.15.99:0/1181331453 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc01d040 con 0x55ea4cd56c60 2024-03-23T23:44:57.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.492+0000 7fbfafe00640 1 -- 172.21.15.99:0/1181331453 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc010be0 con 0x55ea4cd56c60 2024-03-23T23:44:57.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.492+0000 7fbfafe00640 1 -- 172.21.15.99:0/1181331453 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc005470 con 0x55ea4cd56c60 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1181331453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 msgr2=0x55ea4cd57110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd57110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc000f80 tx=0x7fbfbc007ad0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1181331453 shutdown_connections 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd57110 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd54580 0x55ea4cd64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1181331453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd51750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1181331453 >> 172.21.15.99:0/1181331453 conn(0x55ea4cd4dde0 msgr2=0x55ea4cd51040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1181331453 shutdown_connections 2024-03-23T23:44:57.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.493+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1181331453 wait complete. 2024-03-23T23:44:57.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.494+0000 7fbfcd42f740 1 -- start start 2024-03-23T23:44:57.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.494+0000 7fbfcd42f740 1 Processor -- start 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.494+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd63b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.494+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd54580 0x55ea4cd640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.494+0000 7fbfcd42f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400f6c0 con 0x55ea4cd53bc0 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400f060 con 0x55ea4cd54580 2024-03-23T23:44:57.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfcd42f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400c500 con 0x55ea4cd56c60 2024-03-23T23:44:57.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfcd42f740 10 client.0 _ll_register_callbacks cb 0x7fbfb4043040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48704/0 (socket says 172.21.15.99:48704) 2024-03-23T23:44:57.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1400118773 learned_addr learned my addr 172.21.15.99:0/1400118773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3400640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd63b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1400118773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd54580 msgr2=0x55ea4cd640b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:57.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd54580 0x55ea4cd640b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.495+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 msgr2=0x55ea4cd63b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd63b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfc3e00640 1 -- 172.21.15.99:0/1400118773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea4cd60f30 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfc3400640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfc3e00640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007ee0 tx=0x7fbfbc0073b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9061d6d4cd397837 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfad600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc003b70 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfbc010be0 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.496+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbfbc003d10 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.497+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfbc0051f0 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.497+0000 7fbfcd42f740 10 client.25741 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.497+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea4cdc4cf0 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.497+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc010be0 con 0x55ea4cd56c60 2024-03-23T23:44:57.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.497+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbfbc023c80 con 0x55ea4cd56c60 2024-03-23T23:44:57.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 10 client.25741 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 10 client.25741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:44:57.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 20 client.25741 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cdc5350 0x55ea4cdc7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 4 client.25741 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea4ccbeb50 con 0x55ea4cdc5350 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.498+0000 7fbfc3400640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cdc5350 0x55ea4cdc7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.499+0000 7fbfc3400640 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cdc5350 0x55ea4cdc7770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3051f71de967a1 server_cookie=b8525d0d6d6f98a9 in_seq=0 out_seq=0 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.499+0000 7fbfad600640 10 client.25741 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.499+0000 7fbfad600640 1 -- 172.21.15.99:0/1400118773 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea4ccbeb50 con 0x55ea4cdc5350 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.499+0000 7fbfad600640 10 client.25741 handle_command_reply: tid=0 2024-03-23T23:44:57.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.499+0000 7fbfcd42f740 1 client.25741 shutdown 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 msgr2=0x55ea4cd607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007ee0 tx=0x7fbfbc0073b0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 shutdown_connections 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55ea4cdc5350 0x55ea4cdc7770 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfad600640 0 client.25741 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea4cd56c60 0x55ea4cd607f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea4cd54580 0x55ea4cd640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 --2- 172.21.15.99:0/1400118773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea4cd53bc0 0x55ea4cd63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 >> 172.21.15.99:0/1400118773 conn(0x55ea4cd4dde0 msgr2=0x55ea4cd51040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 shutdown_connections 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.500+0000 7fbfcd42f740 1 -- 172.21.15.99:0/1400118773 wait complete. 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.501+0000 7fbfcd42f740 20 client.25741 trim_cache size 0 max 16384 2024-03-23T23:44:57.502 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.501+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 msgr2=0x7fbfa80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.501+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 0x7fbfa80a4d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ea4cd4ed50 tx=0x7fbfb402df90 comp rx=0 tx=0).stop 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.501+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 msgr2=0x7fbfc40704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.501+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 0x7fbfc40704e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e9f0 tx=0x7fbfb000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 shutdown_connections 2024-03-23T23:44:57.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbfa80a2880 0x7fbfa80a4d40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc4105110 0x7fbfc41adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc4102760 0x7fbfc40704e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 --2- 172.21.15.99:0/3887256875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ffdf0 0x7fbfc406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 >> 172.21.15.99:0/3887256875 conn(0x7fbfc40fba10 msgr2=0x7fbfc41030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 shutdown_connections 2024-03-23T23:44:57.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.503+0000 7fbfc9000640 1 -- 172.21.15.99:0/3887256875 wait complete. 2024-03-23T23:44:57.526 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:57.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T23:44:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 -- 172.21.15.99:0/1432957198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 msgr2=0x7feab01096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 --2- 172.21.15.99:0/1432957198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab01096a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009be0 tx=0x7feaa802bf10 comp rx=0 tx=0).stop 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 -- 172.21.15.99:0/1432957198 shutdown_connections 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 --2- 172.21.15.99:0/1432957198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab01096a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 --2- 172.21.15.99:0/1432957198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 0x7feab0106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 --2- 172.21.15.99:0/1432957198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab00ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.847+0000 7feaafe00640 1 -- 172.21.15.99:0/1432957198 >> 172.21.15.99:0/1432957198 conn(0x7feab00fad20 msgr2=0x7feab00fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.848+0000 7feaafe00640 1 -- 172.21.15.99:0/1432957198 shutdown_connections 2024-03-23T23:44:57.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.849+0000 7feaafe00640 1 -- 172.21.15.99:0/1432957198 wait complete. 2024-03-23T23:44:57.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 Processor -- start 2024-03-23T23:44:57.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 -- start start 2024-03-23T23:44:57.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab01025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 0x7feab0100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010d210 con 0x7feab00ff2f0 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010d090 con 0x7feab00ffc50 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.850+0000 7feaafe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010d390 con 0x7feab01071e0 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab01025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48720/0 (socket says 172.21.15.99:48720) 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 -- 172.21.15.99:0/1616733262 learned_addr learned my addr 172.21.15.99:0/1616733262 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaae000640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 0x7feab0100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 -- 172.21.15.99:0/1616733262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 msgr2=0x7feab0100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 0x7feab0100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.851+0000 7feaaf400640 1 -- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 msgr2=0x7feab01025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaaf400640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab01025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaaf400640 1 -- 172.21.15.99:0/1616733262 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab01019b0 con 0x7feab01071e0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaaea00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaaf400640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaa80334c0 tx=0x7feaa80338d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaa7400640 1 -- 172.21.15.99:0/1616733262 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8043020 con 0x7feab01071e0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaa7400640 1 -- 172.21.15.99:0/1616733262 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaa8003c40 con 0x7feab01071e0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.852+0000 7feaa7400640 1 -- 172.21.15.99:0/1616733262 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80347d0 con 0x7feab01071e0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.853+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab01a1910 con 0x7feab01071e0 2024-03-23T23:44:57.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.853+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a1e10 con 0x7feab01071e0 2024-03-23T23:44:57.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.858+0000 7feab824d740 1 Processor -- start 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feaa7400640 1 -- 172.21.15.99:0/1616733262 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7feaa8046020 con 0x7feab01071e0 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 -- start start 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3a200 0x55ff52a3a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 0x55ff52a3b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 0x55ff52a42620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff529962c0 con 0x55ff52a3a200 2024-03-23T23:44:57.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff5293ec90 con 0x55ff52a3b3d0 2024-03-23T23:44:57.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.859+0000 7feab824d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff52992360 con 0x55ff52a3bd90 2024-03-23T23:44:57.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.860+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 0x55ff52a3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.860+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 0x55ff52a3b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38752/0 (socket says 172.21.15.99:38752) 2024-03-23T23:44:57.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.860+0000 7feaaea00640 1 -- 172.21.15.99:0/419939156 learned_addr learned my addr 172.21.15.99:0/419939156 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.860+0000 7feaae000640 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 0x55ff52a42620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.860+0000 7feaae000640 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3a200 0x55ff52a3a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 -- 172.21.15.99:0/419939156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 msgr2=0x55ff52a3b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 0x55ff52a3b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 -- 172.21.15.99:0/419939156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3a200 msgr2=0x55ff52a3a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3a200 0x55ff52a3a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 -- 172.21.15.99:0/419939156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff52963080 con 0x55ff52a3bd90 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.861+0000 7feaae000640 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 0x55ff52a42620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005dc0 tx=0x7fea9c00e390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78c1507fd414891c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.862+0000 7fea9b400640 1 -- 172.21.15.99:0/419939156 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c01d600 con 0x55ff52a3bd90 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.862+0000 7fea9b400640 1 -- 172.21.15.99:0/419939156 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0118c0 con 0x55ff52a3bd90 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.862+0000 7fea9b400640 1 -- 172.21.15.99:0/419939156 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c010960 con 0x55ff52a3bd90 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.863+0000 7feab824d740 1 -- 172.21.15.99:0/419939156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 msgr2=0x55ff52a42620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.863+0000 7feab824d740 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 0x55ff52a42620 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005dc0 tx=0x7fea9c00e390 comp rx=0 tx=0).stop 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.863+0000 7feaa7400640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 0x7fea940a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.863+0000 7feaa7400640 1 -- 172.21.15.99:0/1616733262 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feaa80f16b0 con 0x7feab01071e0 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.864+0000 7feaae000640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 0x7fea940a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 -- 172.21.15.99:0/419939156 shutdown_connections 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3bd90 0x55ff52a42620 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3b3d0 0x55ff52a3b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 --2- 172.21.15.99:0/419939156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3a200 0x55ff52a3a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 -- 172.21.15.99:0/419939156 >> 172.21.15.99:0/419939156 conn(0x55ff52a2cde0 msgr2=0x55ff52a2d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feaae000640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 0x7fea940a4e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005dc0 tx=0x7fea9c00edd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 -- 172.21.15.99:0/419939156 shutdown_connections 2024-03-23T23:44:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.865+0000 7feab824d740 1 -- 172.21.15.99:0/419939156 wait complete. 2024-03-23T23:44:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.866+0000 7feab824d740 1 -- start start 2024-03-23T23:44:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.866+0000 7feab824d740 1 Processor -- start 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 0x55ff52a44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3bd90 0x55ff52a4b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff52a42da0 con 0x55ff52a3b3d0 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff529520d0 con 0x55ff52a3bd90 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff52a42f20 con 0x55ff52a3a200 2024-03-23T23:44:57.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feab824d740 10 client.0 _ll_register_callbacks cb 0x55ff52956e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feaaea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feaaea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48732/0 (socket says 172.21.15.99:48732) 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feaaea00640 1 -- 172.21.15.99:0/1584968325 learned_addr learned my addr 172.21.15.99:0/1584968325 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feaae000640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 0x55ff52a44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.867+0000 7feaaf400640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3bd90 0x55ff52a4b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 -- 172.21.15.99:0/1584968325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3bd90 msgr2=0x55ff52a4b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3bd90 0x55ff52a4b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 -- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 msgr2=0x55ff52a44950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 0x55ff52a44950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 -- 172.21.15.99:0/1584968325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff526aa3f0 con 0x55ff52a3a200 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaae000640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 0x55ff52a44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7feaaea00640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feaa0012840 tx=0x7feaa0012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898325205beab57e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.868+0000 7fea8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0011b50 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feaa0011cf0 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea9c0118c0 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaa001c420 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7feab824d740 10 client.25753 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.869+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ff527747d0 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0118c0 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7feaa00267b0 con 0x55ff52a3a200 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7feab824d740 10 client.25753 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7feab824d740 10 client.25753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:44:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7feab824d740 20 client.25753 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.870+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52a4ff00 0x55ff52a523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.871+0000 7feab824d740 4 client.25753 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.871+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff529a3800 con 0x55ff52a4ff00 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.871+0000 7feaae000640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52a4ff00 0x55ff52a523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.872+0000 7feaae000640 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52a4ff00 0x55ff52a523a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=958ed5a3529f541 server_cookie=970fe890ce9f9f0 in_seq=0 out_seq=0 2024-03-23T23:44:57.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.872+0000 7fea8fe00640 10 client.25753 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:57.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.873+0000 7fea8fe00640 1 -- 172.21.15.99:0/1584968325 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff529a3800 con 0x55ff52a4ff00 2024-03-23T23:44:57.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.873+0000 7fea8fe00640 10 client.25753 handle_command_reply: tid=0 2024-03-23T23:44:57.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.873+0000 7feab824d740 1 client.25753 shutdown 2024-03-23T23:44:57.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 msgr2=0x55ff52a44410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feaa0012840 tx=0x7feaa0012d00 comp rx=0 tx=0).stop 2024-03-23T23:44:57.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 shutdown_connections 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52a4ff00 0x55ff52a523a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3bd90 0x55ff52a4b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3b3d0 0x55ff52a44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 --2- 172.21.15.99:0/1584968325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3a200 0x55ff52a44410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 >> 172.21.15.99:0/1584968325 conn(0x55ff52a2cde0 msgr2=0x55ff52a2c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7fea8fe00640 0 client.25753 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 shutdown_connections 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.874+0000 7feab824d740 1 -- 172.21.15.99:0/1584968325 wait complete. 2024-03-23T23:44:57.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.875+0000 7feab824d740 20 client.25753 trim_cache size 0 max 16384 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.900+0000 7feab824d740 1 Processor -- start 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.900+0000 7feab824d740 1 -- start start 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a65d0 con 0x55ff52a3bd90 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a7f50 con 0x55ff52a3a200 2024-03-23T23:44:57.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feab824d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a6c30 con 0x55ff52a3b3d0 2024-03-23T23:44:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38772/0 (socket says 172.21.15.99:38772) 2024-03-23T23:44:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feaaea00640 1 -- 172.21.15.99:0/3417311963 learned_addr learned my addr 172.21.15.99:0/3417311963 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.901+0000 7feaae000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaf400640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaea00640 1 -- 172.21.15.99:0/3417311963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 msgr2=0x55ff52a4b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaea00640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a4b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaea00640 1 -- 172.21.15.99:0/3417311963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 msgr2=0x55ff52a56c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaea00640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a56c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaaea00640 1 -- 172.21.15.99:0/3417311963 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff52a2d850 con 0x55ff52a3a200 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.902+0000 7feaae000640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a4b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.903+0000 7feaaf400640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:57.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.903+0000 7feaaea00640 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feaa0000c00 tx=0x7feaa00097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909b96f96b3f3a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.903+0000 7fea9aa00640 1 -- 172.21.15.99:0/3417311963 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0011b50 con 0x55ff52a3a200 2024-03-23T23:44:57.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.904+0000 7fea9aa00640 1 -- 172.21.15.99:0/3417311963 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaa0026900 con 0x55ff52a3a200 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.904+0000 7fea9aa00640 1 -- 172.21.15.99:0/3417311963 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa00035f0 con 0x55ff52a3a200 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- 172.21.15.99:0/3417311963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 msgr2=0x55ff52a39320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feaa0000c00 tx=0x7feaa00097a0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- 172.21.15.99:0/3417311963 shutdown_connections 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a4b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 --2- 172.21.15.99:0/3417311963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a39320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- 172.21.15.99:0/3417311963 >> 172.21.15.99:0/3417311963 conn(0x55ff52a2cde0 msgr2=0x55ff52a2c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- 172.21.15.99:0/3417311963 shutdown_connections 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- 172.21.15.99:0/3417311963 wait complete. 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.905+0000 7feab824d740 1 -- start start 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 Processor -- start 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a4ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a78f0 con 0x55ff52a3bd90 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a7290 con 0x55ff52a3a200 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.906+0000 7feab824d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01a4790 con 0x55ff52a3b3d0 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feab824d740 10 client.0 _ll_register_callbacks cb 0x7feaa00098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:57.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a49700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38784/0 (socket says 172.21.15.99:38784) 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaaea00640 1 -- 172.21.15.99:0/2716742827 learned_addr learned my addr 172.21.15.99:0/2716742827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaae000640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaae000640 1 -- 172.21.15.99:0/2716742827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 msgr2=0x55ff52a49700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaae000640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a49700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaae000640 1 -- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 msgr2=0x55ff52a4ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.907+0000 7feaaf400640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a4ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7feaae000640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a4ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7feaae000640 1 -- 172.21.15.99:0/2716742827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff52a4b240 con 0x55ff52a3b3d0 2024-03-23T23:44:57.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7feaaf400640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a4ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:57.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7feaae000640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a49c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e7f0 tx=0x7fea9c01f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801abdea36ebf296 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:57.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7fea8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:57.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.908+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0366f0 con 0x55ff52a3b3d0 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.909+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fea9c036890 con 0x55ff52a3b3d0 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.909+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa0026900 con 0x55ff52a3b3d0 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.909+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea9c036a90 con 0x55ff52a3b3d0 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.909+0000 7feab824d740 10 client.25759 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.909+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ff52abce30 con 0x55ff52a3b3d0 2024-03-23T23:44:57.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.910+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaa0026900 con 0x55ff52a3b3d0 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fea9c00f040 con 0x55ff52a3b3d0 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 10 client.25759 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 10 client.25759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 20 client.25759 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52abd340 0x55ff52abf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:57.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 4 client.25759 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff5299db50 con 0x55ff52abd340 2024-03-23T23:44:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.911+0000 7feaaea00640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52abd340 0x55ff52abf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.912+0000 7feaaea00640 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52abd340 0x55ff52abf730 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=167da1d0ca1f1223 server_cookie=b738efbb1924c62f in_seq=0 out_seq=0 2024-03-23T23:44:57.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.912+0000 7fea8fe00640 10 client.25759 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7fea8fe00640 1 -- 172.21.15.99:0/2716742827 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff5299db50 con 0x55ff52abd340 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7fea8fe00640 10 client.25759 handle_command_reply: tid=0 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 client.25759 shutdown 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 msgr2=0x55ff52a49c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a49c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e7f0 tx=0x7fea9c01f050 comp rx=0 tx=0).stop 2024-03-23T23:44:57.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 shutdown_connections 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55ff52abd340 0x55ff52abf730 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff52a3bd90 0x55ff52a4ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff52a3b3d0 0x55ff52a49c40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 --2- 172.21.15.99:0/2716742827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ff52a3a200 0x55ff52a49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 >> 172.21.15.99:0/2716742827 conn(0x55ff52a2cde0 msgr2=0x55ff52a2b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.913+0000 7fea8fe00640 0 client.25759 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:44:57.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.914+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 shutdown_connections 2024-03-23T23:44:57.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.914+0000 7feab824d740 1 -- 172.21.15.99:0/2716742827 wait complete. 2024-03-23T23:44:57.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.914+0000 7feab824d740 20 client.25759 trim_cache size 0 max 16384 2024-03-23T23:44:57.915 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.915+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 msgr2=0x7fea940a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.915+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 0x7fea940a4e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005dc0 tx=0x7fea9c00edd0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.915+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 msgr2=0x7feab0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.915+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feaa80334c0 tx=0x7feaa80338d0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 shutdown_connections 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fea940a29d0 0x7fea940a4e90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab01071e0 0x7feab0101120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab00ffc50 0x7feab0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 --2- 172.21.15.99:0/1616733262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feab00ff2f0 0x7feab01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:57.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 >> 172.21.15.99:0/1616733262 conn(0x7feab00fad20 msgr2=0x7feab0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 shutdown_connections 2024-03-23T23:44:57.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:57.916+0000 7feaafe00640 1 -- 172.21.15.99:0/1616733262 wait complete. 2024-03-23T23:44:57.939 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:44:57.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 -- 172.21.15.99:0/3710352215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c102000 msgr2=0x7f4f3c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3710352215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c102000 0x7f4f3c1043f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802bc60 comp rx=0 tx=0).stop 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 -- 172.21.15.99:0/3710352215 shutdown_connections 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3710352215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c107260 0x7f4f3c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3710352215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 0x7f4f3c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3710352215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c102000 0x7f4f3c1043f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 -- 172.21.15.99:0/3710352215 >> 172.21.15.99:0/3710352215 conn(0x7f4f3c0fba10 msgr2=0x7f4f3c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.219+0000 7f4f40e00640 1 -- 172.21.15.99:0/3710352215 shutdown_connections 2024-03-23T23:44:58.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.220+0000 7f4f40e00640 1 -- 172.21.15.99:0/3710352215 wait complete. 2024-03-23T23:44:58.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.221+0000 7f4f40e00640 1 Processor -- start 2024-03-23T23:44:58.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.221+0000 7f4f40e00640 1 -- start start 2024-03-23T23:44:58.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 0x7f4f3c195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 0x7f4f3c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10b050 con 0x7f4f3c104930 2024-03-23T23:44:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10aed0 con 0x7f4f3c102000 2024-03-23T23:44:58.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f40e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10b1d0 con 0x7f4f3c107260 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38806/0 (socket says 172.21.15.99:38806) 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.222+0000 7f4f3b400640 1 -- 172.21.15.99:0/3875920932 learned_addr learned my addr 172.21.15.99:0/3875920932 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 0x7f4f3c195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 0x7f4f3c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 -- 172.21.15.99:0/3875920932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 msgr2=0x7f4f3c1955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 -- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 msgr2=0x7f4f3c195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 0x7f4f3c195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.223+0000 7f4f3be00640 1 -- 172.21.15.99:0/3875920932 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c19d200 con 0x7f4f3c107260 2024-03-23T23:44:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.224+0000 7f4f3b400640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.224+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 0x7f4f3c19cb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009fd0 tx=0x7f4f3400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.225+0000 7f4f33e00640 1 -- 172.21.15.99:0/3875920932 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400cd60 con 0x7f4f3c107260 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.225+0000 7f4f33e00640 1 -- 172.21.15.99:0/3875920932 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f34005020 con 0x7f4f3c107260 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.225+0000 7f4f33e00640 1 -- 172.21.15.99:0/3875920932 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34010690 con 0x7f4f3c107260 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.225+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c19d4f0 con 0x7f4f3c107260 2024-03-23T23:44:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.225+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c19d9f0 con 0x7f4f3c107260 2024-03-23T23:44:58.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.230+0000 7f4f45078740 1 Processor -- start 2024-03-23T23:44:58.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.235+0000 7f4f45078740 1 -- start start 2024-03-23T23:44:58.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.235+0000 7f4f33e00640 1 -- 172.21.15.99:0/3875920932 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4f34005540 con 0x7f4f3c107260 2024-03-23T23:44:58.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6612482c0 con 0x55a6612ec200 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6611f0c90 con 0x55a6612ed3d0 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.236+0000 7f4f45078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a661244360 con 0x55a6612edd90 2024-03-23T23:44:58.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.237+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.237+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38818/0 (socket says 172.21.15.99:38818) 2024-03-23T23:44:58.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.237+0000 7f4f3b400640 1 -- 172.21.15.99:0/3468498927 learned_addr learned my addr 172.21.15.99:0/3468498927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:58.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.237+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.238+0000 7f4f33e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 0x7f4f200a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.238+0000 7f4f33e00640 1 -- 172.21.15.99:0/3875920932 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f340cf160 con 0x7f4f3c107260 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.238+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.238+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 0x7f4f200a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 -- 172.21.15.99:0/3468498927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 msgr2=0x55a6612f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 -- 172.21.15.99:0/3468498927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 msgr2=0x55a6612ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 -- 172.21.15.99:0/3468498927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a661215080 con 0x55a6612ed3d0 2024-03-23T23:44:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3b400640 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009690 tx=0x7f4f28005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aba13203194ef07 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.239+0000 7f4f3be00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 0x7f4f200a4d30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005ba0 tx=0x7f4f34005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.240+0000 7f4f30c00640 1 -- 172.21.15.99:0/3468498927 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28043070 con 0x55a6612ed3d0 2024-03-23T23:44:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.240+0000 7f4f30c00640 1 -- 172.21.15.99:0/3468498927 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2802e7a0 con 0x55a6612ed3d0 2024-03-23T23:44:58.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.240+0000 7f4f30c00640 1 -- 172.21.15.99:0/3468498927 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2802cb70 con 0x55a6612ed3d0 2024-03-23T23:44:58.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 -- 172.21.15.99:0/3468498927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 msgr2=0x55a6612ed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009690 tx=0x7f4f28005c00 comp rx=0 tx=0).stop 2024-03-23T23:44:58.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 -- 172.21.15.99:0/3468498927 shutdown_connections 2024-03-23T23:44:58.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612ed850 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 --2- 172.21.15.99:0/3468498927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 -- 172.21.15.99:0/3468498927 >> 172.21.15.99:0/3468498927 conn(0x55a6612e75f0 msgr2=0x55a6612e79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 -- 172.21.15.99:0/3468498927 shutdown_connections 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.241+0000 7f4f45078740 1 -- 172.21.15.99:0/3468498927 wait complete. 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.242+0000 7f4f45078740 1 -- start start 2024-03-23T23:44:58.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.242+0000 7f4f45078740 1 Processor -- start 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ed3d0 0x55a6612f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6612f4d40 con 0x55a6612ed3d0 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6612040d0 con 0x55a6612ec200 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6612f4ec0 con 0x55a6612edd90 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f45078740 10 client.0 _ll_register_callbacks cb 0x55a661208e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38826/0 (socket says 172.21.15.99:38826) 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f3aa00640 1 -- 172.21.15.99:0/1676298690 learned_addr learned my addr 172.21.15.99:0/1676298690 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:58.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.243+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.244+0000 7f4f3aa00640 1 -- 172.21.15.99:0/1676298690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 msgr2=0x55a6612fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.244+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.244+0000 7f4f3aa00640 1 -- 172.21.15.99:0/1676298690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ed3d0 msgr2=0x55a6612f69c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:44:58.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.244+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ed3d0 0x55a6612f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.244+0000 7f4f3aa00640 1 -- 172.21.15.99:0/1676298690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a660f5c3f0 con 0x55a6612ec200 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c013fd0 tx=0x7f4f2c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af354d70ec25182 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f25600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c01a040 con 0x55a6612ec200 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f2c005020 con 0x55a6612ec200 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f2802e7a0 con 0x55a6612ec200 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.245+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f2c027070 con 0x55a6612ec200 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 10 client.25724 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:58.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a6610267d0 con 0x55a6612ec200 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2802e7a0 con 0x55a6612ec200 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4f2c00e5c0 con 0x55a6612ec200 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 10 client.25724 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 20 client.25724 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a6612e2770 0x55a6612e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 4 client.25724 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.246+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a661255800 con 0x55a6612e2770 2024-03-23T23:44:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.247+0000 7f4f3b400640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a6612e2770 0x55a6612e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.247+0000 7f4f3b400640 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a6612e2770 0x55a6612e4bd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75666bfb591c358 server_cookie=bf639e154cb38439 in_seq=0 out_seq=0 2024-03-23T23:44:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.247+0000 7f4f25600640 10 client.25724 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.248+0000 7f4f25600640 1 -- 172.21.15.99:0/1676298690 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a661255800 con 0x55a6612e2770 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.248+0000 7f4f25600640 10 client.25724 handle_command_reply: tid=0 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.248+0000 7f4f45078740 1 client.25724 shutdown 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.248+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 msgr2=0x55a6612f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.248+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c013fd0 tx=0x7f4f2c004730 comp rx=0 tx=0).stop 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 shutdown_connections 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a6612e2770 0x55a6612e4bd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ed3d0 0x55a6612f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 --2- 172.21.15.99:0/1676298690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ec200 0x55a6612f6480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 >> 172.21.15.99:0/1676298690 conn(0x55a6612e75f0 msgr2=0x55a6612ea850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f25600640 0 client.25724 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 shutdown_connections 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 1 -- 172.21.15.99:0/1676298690 wait complete. 2024-03-23T23:44:58.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.249+0000 7f4f45078740 20 client.25724 trim_cache size 0 max 16384 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 Processor -- start 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 -- start start 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612edd90 0x55a6612f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a6500 con 0x55a6612ec200 2024-03-23T23:44:58.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a7e80 con 0x55a6612edd90 2024-03-23T23:44:58.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f45078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a6b60 con 0x55a6612ed3d0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.274+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48828/0 (socket says 172.21.15.99:48828) 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 -- 172.21.15.99:0/1482186636 learned_addr learned my addr 172.21.15.99:0/1482186636 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612edd90 0x55a6612f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 -- 172.21.15.99:0/1482186636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612edd90 msgr2=0x55a6612f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612edd90 0x55a6612f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 -- 172.21.15.99:0/1482186636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 msgr2=0x55a6612f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 -- 172.21.15.99:0/1482186636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6612ece50 con 0x55a6612ed3d0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.275+0000 7f4f3b400640 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f28004c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd9e957996a15f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.276+0000 7f4f27e00640 1 -- 172.21.15.99:0/1482186636 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28039040 con 0x55a6612ed3d0 2024-03-23T23:44:58.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.276+0000 7f4f27e00640 1 -- 172.21.15.99:0/1482186636 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2802e7a0 con 0x55a6612ed3d0 2024-03-23T23:44:58.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.276+0000 7f4f27e00640 1 -- 172.21.15.99:0/1482186636 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28043040 con 0x55a6612ed3d0 2024-03-23T23:44:58.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.277+0000 7f4f45078740 1 -- 172.21.15.99:0/1482186636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 msgr2=0x55a6612fd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.277+0000 7f4f45078740 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f280096c0 tx=0x7f4f28004c90 comp rx=0 tx=0).stop 2024-03-23T23:44:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 -- 172.21.15.99:0/1482186636 shutdown_connections 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612edd90 0x55a6612f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612ed3d0 0x55a6612fd9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 --2- 172.21.15.99:0/1482186636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 -- 172.21.15.99:0/1482186636 >> 172.21.15.99:0/1482186636 conn(0x55a6612e75f0 msgr2=0x55a6612eb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 -- 172.21.15.99:0/1482186636 shutdown_connections 2024-03-23T23:44:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.285+0000 7f4f45078740 1 -- 172.21.15.99:0/1482186636 wait complete. 2024-03-23T23:44:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.286+0000 7f4f45078740 1 -- start start 2024-03-23T23:44:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.286+0000 7f4f45078740 1 Processor -- start 2024-03-23T23:44:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a7820 con 0x55a6612ec200 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a71c0 con 0x55a6612ed3d0 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c1a46c0 con 0x55a6612edd90 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f45078740 10 client.0 _ll_register_callbacks cb 0x7f4f2c00ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f3aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:48844/0 (socket says 172.21.15.99:48844) 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f3aa00640 1 -- 172.21.15.99:0/149202712 learned_addr learned my addr 172.21.15.99:0/149202712 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f3b400640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.287+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.288+0000 7f4f3aa00640 1 -- 172.21.15.99:0/149202712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 msgr2=0x55a6612f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.288+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.288+0000 7f4f3aa00640 1 -- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 msgr2=0x55a6612fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.288+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.288+0000 7f4f3aa00640 1 -- 172.21.15.99:0/149202712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6612fa810 con 0x55a6612ec200 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.289+0000 7f4f3aa00640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612fd420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00a480 tx=0x7f4f2c00afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21381ad24ff1dc13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:58.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.289+0000 7f4f25600640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.289+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00eb80 con 0x55a6612ec200 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.289+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f2c00ed20 con 0x55a6612ec200 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.290+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f2802e7a0 con 0x55a6612ec200 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.290+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f2c040050 con 0x55a6612ec200 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.290+0000 7f4f45078740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:58.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.290+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a6613668b0 con 0x55a6612ec200 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f2802e7a0 con 0x55a6612ec200 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4f2c004870 con 0x55a6612ec200 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 20 client.15579 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a661367290 0x55a6613696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 4 client.15579 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a66124fb50 con 0x55a661367290 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.291+0000 7f4f3b400640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a661367290 0x55a6613696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.292+0000 7f4f3b400640 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a661367290 0x55a6613696b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e906d3b5ddde1d server_cookie=4e81ff0db718f987 in_seq=0 out_seq=0 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.292+0000 7f4f25600640 10 client.15579 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.292+0000 7f4f25600640 1 -- 172.21.15.99:0/149202712 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a66124fb50 con 0x55a661367290 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.292+0000 7f4f25600640 10 client.15579 handle_command_reply: tid=0 2024-03-23T23:44:58.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.292+0000 7f4f45078740 1 client.15579 shutdown 2024-03-23T23:44:58.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.293+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 msgr2=0x55a6612fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.293+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612fd420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00a480 tx=0x7f4f2c00afd0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 shutdown_connections 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a661367290 0x55a6613696b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6612edd90 0x55a6612f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6612ed3d0 0x55a6612fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 --2- 172.21.15.99:0/149202712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6612ec200 0x55a6612fd420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 >> 172.21.15.99:0/149202712 conn(0x55a6612e75f0 msgr2=0x55a6612dd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f25600640 0 client.15579 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:58.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.294+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 shutdown_connections 2024-03-23T23:44:58.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.296+0000 7f4f45078740 1 -- 172.21.15.99:0/149202712 wait complete. 2024-03-23T23:44:58.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.296+0000 7f4f45078740 20 client.15579 trim_cache size 0 max 16384 2024-03-23T23:44:58.296 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:44:58.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 msgr2=0x7f4f200a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 0x7f4f200a4d30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f34005ba0 tx=0x7f4f34005940 comp rx=0 tx=0).stop 2024-03-23T23:44:58.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 msgr2=0x7f4f3c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:58.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 0x7f4f3c19cb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009fd0 tx=0x7f4f3400ee50 comp rx=0 tx=0).stop 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 shutdown_connections 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f4f200a2870 0x7f4f200a4d30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f3c107260 0x7f4f3c19cb90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f3c104930 0x7f4f3c195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 --2- 172.21.15.99:0/3875920932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f3c102000 0x7f4f3c1955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.297+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 >> 172.21.15.99:0/3875920932 conn(0x7f4f3c0fba10 msgr2=0x7f4f3c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.298+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 shutdown_connections 2024-03-23T23:44:58.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:58.298+0000 7f4f40e00640 1 -- 172.21.15.99:0/3875920932 wait complete. 2024-03-23T23:44:58.318 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:57 smithi099 ceph-mon[31715]: pgmap v432: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:57 smithi149 ceph-mon[36083]: pgmap v432: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:58.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:57 smithi174 ceph-mon[37002]: pgmap v432: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:44:59.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T23:44:59.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:44:59.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 -- 172.21.15.99:0/506068705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac102060 msgr2=0x7fd6ac104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 --2- 172.21.15.99:0/506068705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac102060 0x7fd6ac104450 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd694009860 tx=0x7fd69402bb90 comp rx=0 tx=0).stop 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 -- 172.21.15.99:0/506068705 shutdown_connections 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 --2- 172.21.15.99:0/506068705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 0x7fd6ac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 --2- 172.21.15.99:0/506068705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac104990 0x7fd6ac106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 --2- 172.21.15.99:0/506068705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac102060 0x7fd6ac104450 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 -- 172.21.15.99:0/506068705 >> 172.21.15.99:0/506068705 conn(0x7fd6ac0fba10 msgr2=0x7fd6ac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.602+0000 7fd6abe00640 1 -- 172.21.15.99:0/506068705 shutdown_connections 2024-03-23T23:44:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.603+0000 7fd6abe00640 1 -- 172.21.15.99:0/506068705 wait complete. 2024-03-23T23:44:59.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.604+0000 7fd6abe00640 1 Processor -- start 2024-03-23T23:44:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.604+0000 7fd6abe00640 1 -- start start 2024-03-23T23:44:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.604+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac104990 0x7fd6ac19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 0x7fd6ac1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6abe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac10f410 con 0x7fd6ac102060 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6abe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac10f290 con 0x7fd6ac1072c0 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6abe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac10f590 con 0x7fd6ac104990 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39784/0 (socket says 172.21.15.99:39784) 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6aaa00640 1 -- 172.21.15.99:0/2825817288 learned_addr learned my addr 172.21.15.99:0/2825817288 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6aa000640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac104990 0x7fd6ac19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.605+0000 7fd6ab400640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 0x7fd6ac1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.606+0000 7fd6aaa00640 1 -- 172.21.15.99:0/2825817288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac104990 msgr2=0x7fd6ac19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.606+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac104990 0x7fd6ac19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.606+0000 7fd6aaa00640 1 -- 172.21.15.99:0/2825817288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 msgr2=0x7fd6ac1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.606+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 0x7fd6ac1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.606+0000 7fd6aaa00640 1 -- 172.21.15.99:0/2825817288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac1a16f0 con 0x7fd6ac102060 2024-03-23T23:44:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.607+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6940323d0 tx=0x7fd69402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.607+0000 7fd6a3400640 1 -- 172.21.15.99:0/2825817288 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694039040 con 0x7fd6ac102060 2024-03-23T23:44:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.607+0000 7fd6a3400640 1 -- 172.21.15.99:0/2825817288 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69402c9d0 con 0x7fd6ac102060 2024-03-23T23:44:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.607+0000 7fd6a3400640 1 -- 172.21.15.99:0/2825817288 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694004920 con 0x7fd6ac102060 2024-03-23T23:44:59.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.609+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac1a1980 con 0x7fd6ac102060 2024-03-23T23:44:59.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.609+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1a1df0 con 0x7fd6ac102060 2024-03-23T23:44:59.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.616+0000 7fd6a3400640 1 -- 172.21.15.99:0/2825817288 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd694005050 con 0x7fd6ac102060 2024-03-23T23:44:59.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.617+0000 7fd6b49fd740 1 Processor -- start 2024-03-23T23:44:59.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.617+0000 7fd6b49fd740 1 -- start start 2024-03-23T23:44:59.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6a3400640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 0x7fd6900a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1e67eb0 con 0x5565d1f0aac0 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1e10a60 con 0x5565d1f10620 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1e63f50 con 0x5565d1f0b4b0 2024-03-23T23:44:59.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.619+0000 7fd6a3400640 1 -- 172.21.15.99:0/2825817288 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd6940ef900 con 0x7fd6ac102060 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59252/0 (socket says 172.21.15.99:59252) 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39788/0 (socket says 172.21.15.99:39788) 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 -- 172.21.15.99:0/1664038369 learned_addr learned my addr 172.21.15.99:0/1664038369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6aa000640 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 -- 172.21.15.99:0/1664038369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 msgr2=0x5565d1f0ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f0ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 -- 172.21.15.99:0/1664038369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 msgr2=0x5565d1f0aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.620+0000 7fd6ab400640 1 -- 172.21.15.99:0/1664038369 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d1c447e0 con 0x5565d1f10620 2024-03-23T23:44:59.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.621+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:59.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.621+0000 7fd6ab400640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 0x7fd6900a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.622+0000 7fd6ab400640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 0x7fd6900a4dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4008000 tx=0x7fd6a4017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.622+0000 7fd6ab400640 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009d50 tx=0x7fd6a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4429aef50899dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.622+0000 7fd69be00640 1 -- 172.21.15.99:0/1664038369 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4020030 con 0x5565d1f10620 2024-03-23T23:44:59.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.623+0000 7fd69be00640 1 -- 172.21.15.99:0/1664038369 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a400e8a0 con 0x5565d1f10620 2024-03-23T23:44:59.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.623+0000 7fd69be00640 1 -- 172.21.15.99:0/1664038369 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401a870 con 0x5565d1f10620 2024-03-23T23:44:59.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 -- 172.21.15.99:0/1664038369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 msgr2=0x5565d1f12ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009d50 tx=0x7fd6a4011810 comp rx=0 tx=0).stop 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 -- 172.21.15.99:0/1664038369 shutdown_connections 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f12ab0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/1664038369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0aac0 0x5565d1f0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 -- 172.21.15.99:0/1664038369 >> 172.21.15.99:0/1664038369 conn(0x5565d1f06030 msgr2=0x5565d1f06430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 -- 172.21.15.99:0/1664038369 shutdown_connections 2024-03-23T23:44:59.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.624+0000 7fd6b49fd740 1 -- 172.21.15.99:0/1664038369 wait complete. 2024-03-23T23:44:59.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.625+0000 7fd6b49fd740 1 -- start start 2024-03-23T23:44:59.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.625+0000 7fd6b49fd740 1 Processor -- start 2024-03-23T23:44:59.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f1c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1f135d0 con 0x5565d1f0b4b0 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1e23eb0 con 0x5565d1f10620 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d1f13750 con 0x5565d1f0aac0 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6b49fd740 10 client.0 _ll_register_callbacks cb 0x5565d1e18150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53288/0 (socket says 172.21.15.99:53288) 2024-03-23T23:44:59.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.626+0000 7fd6aaa00640 1 -- 172.21.15.99:0/900892280 learned_addr learned my addr 172.21.15.99:0/900892280 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6ab400640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f1c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 -- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 msgr2=0x5565d1f1c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f1c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 -- 172.21.15.99:0/900892280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 msgr2=0x5565d1f150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 -- 172.21.15.99:0/900892280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d1ca4f70 con 0x5565d1f0aac0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6ab400640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f1c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.627+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6940bbfb0 tx=0x7fd694005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755e41a528d82954 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd698c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940bc2b0 con 0x5565d1f0aac0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd6940ed490 con 0x5565d1f0aac0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6a400e8a0 con 0x5565d1f0aac0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6940c0400 con 0x5565d1f0aac0 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.628+0000 7fd6b49fd740 10 client.25789 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:59.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.629+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565d1b3c170 con 0x5565d1f0aac0 2024-03-23T23:44:59.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.629+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a400e8a0 con 0x5565d1f0aac0 2024-03-23T23:44:59.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd69403c070 con 0x5565d1f0aac0 2024-03-23T23:44:59.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 10 client.25789 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:59.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:59.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 20 client.25789 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1f208f0 0x5565d1f22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 4 client.25789 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565d1e753f0 con 0x5565d1f208f0 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.630+0000 7fd6aa000640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1f208f0 0x5565d1f22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.631+0000 7fd6aa000640 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1f208f0 0x5565d1f22d40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511fa790a5e6bb14 server_cookie=42f6454be0a6f59b in_seq=0 out_seq=0 2024-03-23T23:44:59.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.631+0000 7fd698c00640 10 client.25789 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.632+0000 7fd698c00640 1 -- 172.21.15.99:0/900892280 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565d1e753f0 con 0x5565d1f208f0 2024-03-23T23:44:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.632+0000 7fd698c00640 10 client.25789 handle_command_reply: tid=0 2024-03-23T23:44:59.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.632+0000 7fd6b49fd740 1 client.25789 shutdown 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.632+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 msgr2=0x5565d1f14b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.632+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6940bbfb0 tx=0x7fd694005f90 comp rx=0 tx=0).stop 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 shutdown_connections 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1f208f0 0x5565d1f22d40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f1c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/900892280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f14b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 >> 172.21.15.99:0/900892280 conn(0x5565d1f06030 msgr2=0x5565d1efca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd698c00640 0 client.25789 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 shutdown_connections 2024-03-23T23:44:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 1 -- 172.21.15.99:0/900892280 wait complete. 2024-03-23T23:44:59.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.633+0000 7fd6b49fd740 20 client.25789 trim_cache size 0 max 16384 2024-03-23T23:44:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.666+0000 7fd6b49fd740 1 Processor -- start 2024-03-23T23:44:59.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.666+0000 7fd6b49fd740 1 -- start start 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f0bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a6620 con 0x5565d1f0b4b0 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a7fa0 con 0x5565d1f10620 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a6c80 con 0x5565d1f0aac0 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.667+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59282/0 (socket says 172.21.15.99:59282) 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 -- 172.21.15.99:0/4254610604 learned_addr learned my addr 172.21.15.99:0/4254610604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f0bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6aa000640 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 -- 172.21.15.99:0/4254610604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 msgr2=0x5565d1f0bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f0bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 -- 172.21.15.99:0/4254610604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 msgr2=0x5565d1f1c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.668+0000 7fd6ab400640 1 -- 172.21.15.99:0/4254610604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d1efbfa0 con 0x5565d1f10620 2024-03-23T23:44:59.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.669+0000 7fd6ab400640 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009e70 tx=0x7fd6a4009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcd1c893ee4b8c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.669+0000 7fd69b400640 1 -- 172.21.15.99:0/4254610604 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4020040 con 0x5565d1f10620 2024-03-23T23:44:59.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.669+0000 7fd69b400640 1 -- 172.21.15.99:0/4254610604 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4004340 con 0x5565d1f10620 2024-03-23T23:44:59.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.669+0000 7fd69b400640 1 -- 172.21.15.99:0/4254610604 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401aa40 con 0x5565d1f10620 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.670+0000 7fd6b49fd740 1 -- 172.21.15.99:0/4254610604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 msgr2=0x5565d1f0c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009e70 tx=0x7fd6a4009790 comp rx=0 tx=0).stop 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 -- 172.21.15.99:0/4254610604 shutdown_connections 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f10620 0x5565d1f0c490 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/4254610604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0aac0 0x5565d1f0bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 -- 172.21.15.99:0/4254610604 >> 172.21.15.99:0/4254610604 conn(0x5565d1f06030 msgr2=0x5565d1f09b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 -- 172.21.15.99:0/4254610604 shutdown_connections 2024-03-23T23:44:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.671+0000 7fd6b49fd740 1 -- 172.21.15.99:0/4254610604 wait complete. 2024-03-23T23:44:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.672+0000 7fd6b49fd740 1 -- start start 2024-03-23T23:44:59.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 Processor -- start 2024-03-23T23:44:59.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f0aac0 0x5565d1f1b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 0x5565d1f19fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a7940 con 0x5565d1f10620 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a72e0 con 0x5565d1f0aac0 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1a47e0 con 0x5565d1f0b4b0 2024-03-23T23:44:59.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.673+0000 7fd6b49fd740 10 client.0 _ll_register_callbacks cb 0x7fd6940056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6ab400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 0x5565d1f19fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53316/0 (socket says 172.21.15.99:53316) 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 -- 172.21.15.99:0/2491381195 learned_addr learned my addr 172.21.15.99:0/2491381195 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f0aac0 0x5565d1f1b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 -- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f0aac0 msgr2=0x5565d1f1b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f0aac0 0x5565d1f1b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 -- 172.21.15.99:0/2491381195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 msgr2=0x5565d1f19fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 0x5565d1f19fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6aa000640 1 -- 172.21.15.99:0/2491381195 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d1f1a7e0 con 0x5565d1f0b4b0 2024-03-23T23:44:59.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.674+0000 7fd6ab400640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 0x5565d1f19fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:44:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd6aa000640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac069300 tx=0x7fd69c012920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245112227bb78e06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd698c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:44:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c023020 con 0x5565d1f0b4b0 2024-03-23T23:44:59.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd69c01f890 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6a4004340 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.675+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd69c02e050 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.676+0000 7fd6b49fd740 10 client.25795 fetch_fsmap learned FSMap version 27 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.676+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565d1efd800 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.676+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4004340 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd69c010c40 con 0x5565d1f0b4b0 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:44:59.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 20 client.25795 populate_metadata read hostname 'smithi099' 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1efe1b0 0x5565d1f005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 4 client.25795 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5565d1f230d0 con 0x5565d1efe1b0 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.677+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1efe1b0 0x5565d1f005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.678+0000 7fd6aaa00640 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1efe1b0 0x5565d1f005d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2bd53e68cb4f26 server_cookie=86501b2b511130be in_seq=0 out_seq=0 2024-03-23T23:44:59.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.678+0000 7fd698c00640 10 client.25795 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.679+0000 7fd698c00640 1 -- 172.21.15.99:0/2491381195 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5565d1f230d0 con 0x5565d1efe1b0 2024-03-23T23:44:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.679+0000 7fd698c00640 10 client.25795 handle_command_reply: tid=0 2024-03-23T23:44:59.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.679+0000 7fd6b49fd740 1 client.25795 shutdown 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 msgr2=0x5565d1f1bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac069300 tx=0x7fd69c012920 comp rx=0 tx=0).stop 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 shutdown_connections 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5565d1efe1b0 0x5565d1f005d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565d1f10620 0x5565d1f19fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565d1f0b4b0 0x5565d1f1bee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 --2- 172.21.15.99:0/2491381195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565d1f0aac0 0x5565d1f1b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 >> 172.21.15.99:0/2491381195 conn(0x5565d1f06030 msgr2=0x5565d1efc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd698c00640 0 client.25795 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 shutdown_connections 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.680+0000 7fd6b49fd740 1 -- 172.21.15.99:0/2491381195 wait complete. 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.681+0000 7fd6b49fd740 20 client.25795 trim_cache size 0 max 16384 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-23T23:44:59.681 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "6ec69f38-afbb-4cc6-8392-89c059ca0abc", 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 msgr2=0x7fd6900a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 0x7fd6900a4dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4008000 tx=0x7fd6a4017000 comp rx=0 tx=0).stop 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 msgr2=0x7fd6ac199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6940323d0 tx=0x7fd69402c7e0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 shutdown_connections 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6900a2900 0x7fd6900a4dc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6ac1072c0 0x7fd6ac1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6ac104990 0x7fd6ac19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 --2- 172.21.15.99:0/2825817288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6ac102060 0x7fd6ac199ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.682+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 >> 172.21.15.99:0/2825817288 conn(0x7fd6ac0fba10 msgr2=0x7fd6ac1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.683+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 shutdown_connections 2024-03-23T23:44:59.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.683+0000 7fd6abe00640 1 -- 172.21.15.99:0/2825817288 wait complete. 2024-03-23T23:44:59.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:44:59.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:44:59.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.992+0000 7f490cc00640 1 -- 172.21.15.99:0/3249668811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 msgr2=0x7f49081096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.992+0000 7f490cc00640 1 --2- 172.21.15.99:0/3249668811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081096b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4900009ca0 tx=0x7f490002c120 comp rx=0 tx=0).stop 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 -- 172.21.15.99:0/3249668811 shutdown_connections 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 --2- 172.21.15.99:0/3249668811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081096b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 --2- 172.21.15.99:0/3249668811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908104930 0x7f4908106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 --2- 172.21.15.99:0/3249668811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908102000 0x7f49081043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 -- 172.21.15.99:0/3249668811 >> 172.21.15.99:0/3249668811 conn(0x7f49080fba10 msgr2=0x7f49080fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 -- 172.21.15.99:0/3249668811 shutdown_connections 2024-03-23T23:44:59.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.993+0000 7f490cc00640 1 -- 172.21.15.99:0/3249668811 wait complete. 2024-03-23T23:44:59.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.994+0000 7f490cc00640 1 Processor -- start 2024-03-23T23:44:59.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.994+0000 7f490cc00640 1 -- start start 2024-03-23T23:44:59.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 0x7f490806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 0x7f490806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f4906a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 0x7f490806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f4906a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 0x7f490806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59306/0 (socket says 172.21.15.99:59306) 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490810d290 con 0x7f4908102000 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490810d110 con 0x7f4908104930 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f490cc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490810d410 con 0x7f4908107260 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f4906a00640 1 -- 172.21.15.99:0/1494626232 learned_addr learned my addr 172.21.15.99:0/1494626232 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:44:59.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.995+0000 7f4907e00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907e00640 1 -- 172.21.15.99:0/1494626232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 msgr2=0x7f490806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907e00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 0x7f490806e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907e00640 1 -- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 msgr2=0x7f490806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907400640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 0x7f490806de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907e00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 0x7f490806de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907e00640 1 -- 172.21.15.99:0/1494626232 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49080734f0 con 0x7f4908107260 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.996+0000 7f4907400640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 0x7f490806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:44:59.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.997+0000 7f4907e00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081ae1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4900037040 tx=0x7f4900035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:44:59.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.997+0000 7f48ffe00640 1 -- 172.21.15.99:0/1494626232 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900039040 con 0x7f4908107260 2024-03-23T23:44:59.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.997+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49081ae6e0 con 0x7f4908107260 2024-03-23T23:44:59.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.997+0000 7f48ffe00640 1 -- 172.21.15.99:0/1494626232 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4900044070 con 0x7f4908107260 2024-03-23T23:44:59.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.997+0000 7f48ffe00640 1 -- 172.21.15.99:0/1494626232 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900004430 con 0x7f4908107260 2024-03-23T23:44:59.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:44:59.999+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49081aeb60 con 0x7f4908107260 2024-03-23T23:45:00.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.003+0000 7f4911025740 1 Processor -- start 2024-03-23T23:45:00.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.003+0000 7f48ffe00640 1 -- 172.21.15.99:0/1494626232 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f49000045d0 con 0x7f4908107260 2024-03-23T23:45:00.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f4911025740 1 -- start start 2024-03-23T23:45:00.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f48ffe00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 0x7f48ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f4911025740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f48ffe00640 1 -- 172.21.15.99:0/1494626232 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f49000f0fe0 con 0x7f4908107260 2024-03-23T23:45:00.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f4911025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.005+0000 7f4911025740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4911025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564808646250 con 0x5648086e1a40 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4911025740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648085eea30 con 0x5648086e6bb0 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4911025740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648086422f0 con 0x5648086e1050 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4906a00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 0x7f48ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4907400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4907400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53342/0 (socket says 172.21.15.99:53342) 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4907400640 1 -- 172.21.15.99:0/424096082 learned_addr learned my addr 172.21.15.99:0/424096082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4906a00640 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.006+0000 7f4907e00640 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 -- 172.21.15.99:0/424096082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 msgr2=0x5648086e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 -- 172.21.15.99:0/424096082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 msgr2=0x5648086e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 -- 172.21.15.99:0/424096082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648085d9690 con 0x5648086e1050 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4906a00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 0x7f48ec0a4d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b6c0 tx=0x7f48f400bbb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.007+0000 7f4907400640 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48f800a9c0 tx=0x7f48f800ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a994eb4b39761fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.008+0000 7f48f3e00640 1 -- 172.21.15.99:0/424096082 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8015070 con 0x5648086e1050 2024-03-23T23:45:00.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.008+0000 7f48f3e00640 1 -- 172.21.15.99:0/424096082 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f800cc60 con 0x5648086e1050 2024-03-23T23:45:00.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.008+0000 7f48f3e00640 1 -- 172.21.15.99:0/424096082 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8015070 con 0x5648086e1050 2024-03-23T23:45:00.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- 172.21.15.99:0/424096082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 msgr2=0x5648086e1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48f800a9c0 tx=0x7f48f800ae80 comp rx=0 tx=0).stop 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- 172.21.15.99:0/424096082 shutdown_connections 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 --2- 172.21.15.99:0/424096082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086e1470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- 172.21.15.99:0/424096082 >> 172.21.15.99:0/424096082 conn(0x5648086a9200 msgr2=0x5648086dc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- 172.21.15.99:0/424096082 shutdown_connections 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- 172.21.15.99:0/424096082 wait complete. 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.009+0000 7f4911025740 1 -- start start 2024-03-23T23:45:00.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 Processor -- start 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 0x5648086eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648086e9b60 con 0x5648086e1050 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564808601e80 con 0x5648086e6bb0 2024-03-23T23:45:00.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648086e9ce0 con 0x5648086e1a40 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.010+0000 7f4911025740 10 client.0 _ll_register_callbacks cb 0x564808614130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53356/0 (socket says 172.21.15.99:53356) 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 -- 172.21.15.99:0/1939681588 learned_addr learned my addr 172.21.15.99:0/1939681588 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4907e00640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4907400640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 0x5648086eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 -- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 msgr2=0x5648086f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 -- 172.21.15.99:0/1939681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 msgr2=0x5648086eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 0x5648086eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4906a00640 1 -- 172.21.15.99:0/1939681588 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648084247d0 con 0x5648086e1a40 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.011+0000 7f4907400640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 0x5648086eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f4906a00640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48f400be70 tx=0x7f48f400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29bba16713d50af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f48f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4011a70 con 0x5648086e1a40 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48f800cc60 con 0x5648086e1a40 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48f4011c10 con 0x5648086e1a40 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f4011e10 con 0x5648086e1a40 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f4911025740 10 client.25807 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56480835a3f0 con 0x5648086e1a40 2024-03-23T23:45:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.012+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f800cc60 con 0x5648086e1a40 2024-03-23T23:45:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f48f401f070 con 0x5648086e1a40 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 10 client.25807 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 10 client.25807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 20 client.25807 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648086f6f30 0x5648086f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 4 client.25807 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564808653790 con 0x5648086f6f30 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.013+0000 7f4907400640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648086f6f30 0x5648086f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.014+0000 7f4907400640 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648086f6f30 0x5648086f93f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f09256346f032f server_cookie=2fe498b2d238712b in_seq=0 out_seq=0 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.014+0000 7f48f0c00640 10 client.25807 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.014+0000 7f48f0c00640 1 -- 172.21.15.99:0/1939681588 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564808653790 con 0x5648086f6f30 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.014+0000 7f48f0c00640 10 client.25807 handle_command_reply: tid=0 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.014+0000 7f4911025740 1 client.25807 shutdown 2024-03-23T23:45:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 msgr2=0x5648086eb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f48f400be70 tx=0x7f48f400b410 comp rx=0 tx=0).stop 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 shutdown_connections 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648086f6f30 0x5648086f93f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086eb630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 --2- 172.21.15.99:0/1939681588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1050 0x5648086eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 >> 172.21.15.99:0/1939681588 conn(0x5648086a9200 msgr2=0x5648086dbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f48f0c00640 0 client.25807 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 shutdown_connections 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 1 -- 172.21.15.99:0/1939681588 wait complete. 2024-03-23T23:45:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.015+0000 7f4911025740 20 client.25807 trim_cache size 0 max 16384 2024-03-23T23:45:00.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 Processor -- start 2024-03-23T23:45:00.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 -- start start 2024-03-23T23:45:00.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b7660 con 0x5648086e1a40 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b8fe0 con 0x5648086e6bb0 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.032+0000 7f4911025740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b7cc0 con 0x5648086e1050 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4907400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4907400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53360/0 (socket says 172.21.15.99:53360) 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4907400640 1 -- 172.21.15.99:0/2657660895 learned_addr learned my addr 172.21.15.99:0/2657660895 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4906a00640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4907400640 1 -- 172.21.15.99:0/2657660895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 msgr2=0x5648086dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.033+0000 7f4907e00640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4907400640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4907400640 1 -- 172.21.15.99:0/2657660895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 msgr2=0x5648086f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4907400640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4907400640 1 -- 172.21.15.99:0/2657660895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56480831a0f0 con 0x5648086e1050 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4907e00640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:00.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.034+0000 7f4906a00640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:00.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.035+0000 7f4907400640 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f48f8004790 tx=0x7f48f8004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f590b7f80dca97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:00.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.035+0000 7f48f3400640 1 -- 172.21.15.99:0/2657660895 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8015040 con 0x5648086e1050 2024-03-23T23:45:00.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.035+0000 7f48f3400640 1 -- 172.21.15.99:0/2657660895 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f804c070 con 0x5648086e1050 2024-03-23T23:45:00.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.035+0000 7f48f3400640 1 -- 172.21.15.99:0/2657660895 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8011770 con 0x5648086e1050 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 -- 172.21.15.99:0/2657660895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 msgr2=0x5648086d7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f48f8004790 tx=0x7f48f8004700 comp rx=0 tx=0).stop 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 -- 172.21.15.99:0/2657660895 shutdown_connections 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e6bb0 0x5648086dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e1a40 0x5648086f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 --2- 172.21.15.99:0/2657660895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1050 0x5648086d7920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 -- 172.21.15.99:0/2657660895 >> 172.21.15.99:0/2657660895 conn(0x5648086a9200 msgr2=0x5648086e2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 -- 172.21.15.99:0/2657660895 shutdown_connections 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.036+0000 7f4911025740 1 -- 172.21.15.99:0/2657660895 wait complete. 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 -- start start 2024-03-23T23:45:00.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 Processor -- start 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e6bb0 0x5648086f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b8980 con 0x5648086e6bb0 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b8320 con 0x5648086e1050 2024-03-23T23:45:00.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081b5820 con 0x5648086e1a40 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.037+0000 7f4911025740 10 client.0 _ll_register_callbacks cb 0x7f48f4012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:59354/0 (socket says 172.21.15.99:59354) 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 -- 172.21.15.99:0/3403330808 learned_addr learned my addr 172.21.15.99:0/3403330808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4906a00640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 -- 172.21.15.99:0/3403330808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 msgr2=0x5648086f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086f0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 -- 172.21.15.99:0/3403330808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e6bb0 msgr2=0x5648086f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e6bb0 0x5648086f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.038+0000 7f4907400640 1 -- 172.21.15.99:0/3403330808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648086f2000 con 0x5648086e1050 2024-03-23T23:45:00.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f4907400640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c520 tx=0x7f48f800cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fbae684302aea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:00.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f48f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:00.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8015040 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48f8005970 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48f804c070 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f8047620 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f4911025740 10 client.25757 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.039+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564808752ec0 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48f804c070 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f48f8052460 con 0x5648086e1050 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f4911025740 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f4911025740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f4911025740 20 client.25757 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.040+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648087534d0 0x5648087558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.041+0000 7f4911025740 4 client.25757 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:45:00.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.041+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56480864dae0 con 0x5648087534d0 2024-03-23T23:45:00.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.041+0000 7f4906a00640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648087534d0 0x5648087558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:00.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f48f0c00640 10 client.25757 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:00.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4906a00640 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648087534d0 0x5648087558f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beccd31df7403193 server_cookie=7c3a8c965edf6ead in_seq=0 out_seq=0 2024-03-23T23:45:00.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f48f0c00640 1 -- 172.21.15.99:0/3403330808 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56480864dae0 con 0x5648087534d0 2024-03-23T23:45:00.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f48f0c00640 10 client.25757 handle_command_reply: tid=0 2024-03-23T23:45:00.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 client.25757 shutdown 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 msgr2=0x5648086f0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c520 tx=0x7f48f800cf10 comp rx=0 tx=0).stop 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 shutdown_connections 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648087534d0 0x5648087558f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648086e6bb0 0x5648086f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f48f0c00640 0 client.25757 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648086e1a40 0x5648086f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 --2- 172.21.15.99:0/3403330808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648086e1050 0x5648086f0510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.043+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 >> 172.21.15.99:0/3403330808 conn(0x5648086a9200 msgr2=0x5648086e23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:00.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.044+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 shutdown_connections 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.044+0000 7f4911025740 1 -- 172.21.15.99:0/3403330808 wait complete. 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.044+0000 7f4911025740 20 client.25757 trim_cache size 0 max 16384 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "6ec69f38-afbb-4cc6-8392-89c059ca0abc": { 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "6ec69f38-afbb-4cc6-8392-89c059ca0abc", 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:45:00.045 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:45:00.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.045+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 msgr2=0x7f48ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.045+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 0x7f48ec0a4d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b6c0 tx=0x7f48f400bbb0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.045+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 msgr2=0x7f49081ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:00.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.045+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081ae1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4900037040 tx=0x7f4900035d10 comp rx=0 tx=0).stop 2024-03-23T23:45:00.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 shutdown_connections 2024-03-23T23:45:00.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f48ec0a2880 0x7f48ec0a4d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4908107260 0x7f49081ae1a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4908104930 0x7f490806e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 --2- 172.21.15.99:0/1494626232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4908102000 0x7f490806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:00.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 >> 172.21.15.99:0/1494626232 conn(0x7f49080fba10 msgr2=0x7f49081016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:00.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 shutdown_connections 2024-03-23T23:45:00.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:00.049+0000 7f490cc00640 1 -- 172.21.15.99:0/1494626232 wait complete. 2024-03-23T23:45:00.066 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:44:59 smithi099 ceph-mon[31715]: pgmap v433: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T23:45:00.066 INFO:tasks.cephfs.filesystem:scrub status for tag:6ec69f38-afbb-4cc6-8392-89c059ca0abc - {'path': '/', 'tag': '6ec69f38-afbb-4cc6-8392-89c059ca0abc', 'options': 'recursive,force'} 2024-03-23T23:45:00.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:44:59 smithi149 ceph-mon[36083]: pgmap v433: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T23:45:00.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:44:59 smithi174 ceph-mon[37002]: pgmap v433: 97 pgs: 97 active+clean; 673 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T23:45:01.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:00 smithi099 ceph-mon[31715]: scrub queued for path: / 2024-03-23T23:45:01.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:00 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:01.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:00 smithi099 ceph-mon[31715]: scrub summary: active paths [/] 2024-03-23T23:45:01.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:00 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:00 smithi149 ceph-mon[36083]: scrub queued for path: / 2024-03-23T23:45:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:00 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:00 smithi149 ceph-mon[36083]: scrub summary: active paths [/] 2024-03-23T23:45:01.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:00 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:01.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:00 smithi174 ceph-mon[37002]: scrub queued for path: / 2024-03-23T23:45:01.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:00 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:01.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:00 smithi174 ceph-mon[37002]: scrub summary: active paths [/] 2024-03-23T23:45:01.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:00 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:02.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:01 smithi099 ceph-mon[31715]: pgmap v434: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:02.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:01 smithi149 ceph-mon[36083]: pgmap v434: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:02.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:01 smithi174 ceph-mon[37002]: pgmap v434: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:04.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:03 smithi099 ceph-mon[31715]: pgmap v435: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T23:45:04.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:03 smithi099 ceph-mon[31715]: scrub summary: idle 2024-03-23T23:45:04.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:03 smithi149 ceph-mon[36083]: pgmap v435: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T23:45:04.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:03 smithi149 ceph-mon[36083]: scrub summary: idle 2024-03-23T23:45:04.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:03 smithi174 ceph-mon[37002]: pgmap v435: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-23T23:45:04.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:03 smithi174 ceph-mon[37002]: scrub summary: idle 2024-03-23T23:45:06.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:05 smithi099 ceph-mon[31715]: pgmap v436: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:06.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:05 smithi149 ceph-mon[36083]: pgmap v436: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:06.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:05 smithi174 ceph-mon[37002]: pgmap v436: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:08.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:07 smithi099 ceph-mon[31715]: pgmap v437: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:08.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:07 smithi149 ceph-mon[36083]: pgmap v437: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:08.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:07 smithi174 ceph-mon[37002]: pgmap v437: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:10.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:09 smithi099 ceph-mon[31715]: pgmap v438: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:10.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:09 smithi149 ceph-mon[36083]: pgmap v438: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:10.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:09 smithi174 ceph-mon[37002]: pgmap v438: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 -- 172.21.15.99:0/4066727455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 msgr2=0x7fc010104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 --2- 172.21.15.99:0/4066727455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc010104360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc000009900 tx=0x7fc00002bc50 comp rx=0 tx=0).stop 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 -- 172.21.15.99:0/4066727455 shutdown_connections 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 --2- 172.21.15.99:0/4066727455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 0x7fc010109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 --2- 172.21.15.99:0/4066727455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc010106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 --2- 172.21.15.99:0/4066727455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc010104360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 -- 172.21.15.99:0/4066727455 >> 172.21.15.99:0/4066727455 conn(0x7fc0100fb980 msgr2=0x7fc0100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 -- 172.21.15.99:0/4066727455 shutdown_connections 2024-03-23T23:45:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.538+0000 7fc015600640 1 -- 172.21.15.99:0/4066727455 wait complete. 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.539+0000 7fc015600640 1 Processor -- start 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 -- start start 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc0100fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 0x7fc010100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01010d230 con 0x7fc0101048a0 2024-03-23T23:45:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01010d0b0 con 0x7fc010101f70 2024-03-23T23:45:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc015600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01010d3b0 con 0x7fc0101071d0 2024-03-23T23:45:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38714/0 (socket says 172.21.15.99:38714) 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.540+0000 7fc00fe00640 1 -- 172.21.15.99:0/3538723476 learned_addr learned my addr 172.21.15.99:0/3538723476 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.541+0000 7fc014c00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 0x7fc010100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.541+0000 7fc00f400640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc0100fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.541+0000 7fc00fe00640 1 -- 172.21.15.99:0/3538723476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 msgr2=0x7fc010100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.541+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 0x7fc010100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.541+0000 7fc00fe00640 1 -- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 msgr2=0x7fc0100fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.542+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc0100fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.542+0000 7fc00fe00640 1 -- 172.21.15.99:0/3538723476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc010100cf0 con 0x7fc010101f70 2024-03-23T23:45:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.542+0000 7fc00f400640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc0100fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.542+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc00002c800 tx=0x7fc000005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.543+0000 7fc00cc00640 1 -- 172.21.15.99:0/3538723476 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000041070 con 0x7fc010101f70 2024-03-23T23:45:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.543+0000 7fc00cc00640 1 -- 172.21.15.99:0/3538723476 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0000054e0 con 0x7fc010101f70 2024-03-23T23:45:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.543+0000 7fc00cc00640 1 -- 172.21.15.99:0/3538723476 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000034440 con 0x7fc010101f70 2024-03-23T23:45:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.543+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0101a1940 con 0x7fc010101f70 2024-03-23T23:45:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.543+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101a1e40 con 0x7fc010101f70 2024-03-23T23:45:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.549+0000 7fc0199ec740 1 Processor -- start 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.549+0000 7fc0199ec740 1 -- start start 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f85a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ade0 0x55e07f85f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f81d3b0 con 0x55e07f85a3f0 2024-03-23T23:45:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f7bf830 con 0x55e07f85ff50 2024-03-23T23:45:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.550+0000 7fc0199ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f7685d0 con 0x55e07f85ade0 2024-03-23T23:45:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.551+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.551+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38720/0 (socket says 172.21.15.99:38720) 2024-03-23T23:45:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.551+0000 7fc00f400640 1 -- 172.21.15.99:0/3890499190 learned_addr learned my addr 172.21.15.99:0/3890499190 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.551+0000 7fc014c00640 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f85a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 -- 172.21.15.99:0/3890499190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ade0 msgr2=0x55e07f85f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ade0 0x55e07f85f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 -- 172.21.15.99:0/3890499190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 msgr2=0x55e07f85a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f85a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 -- 172.21.15.99:0/3890499190 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07f7802c0 con 0x55e07f85ff50 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc014c00640 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f85a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.552+0000 7fc00f400640 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009de0 tx=0x7fbffc00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745f37b85d0b4472 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.553+0000 7fc005200640 1 -- 172.21.15.99:0/3890499190 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc017070 con 0x55e07f85ff50 2024-03-23T23:45:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.553+0000 7fc005200640 1 -- 172.21.15.99:0/3890499190 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbffc013950 con 0x55e07f85ff50 2024-03-23T23:45:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.553+0000 7fc005200640 1 -- 172.21.15.99:0/3890499190 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc012a40 con 0x55e07f85ff50 2024-03-23T23:45:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 -- 172.21.15.99:0/3890499190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 msgr2=0x55e07f8623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009de0 tx=0x7fbffc00e5a0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 -- 172.21.15.99:0/3890499190 shutdown_connections 2024-03-23T23:45:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f8623e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ade0 0x55e07f85f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.554+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3890499190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f85a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.555+0000 7fc0199ec740 1 -- 172.21.15.99:0/3890499190 >> 172.21.15.99:0/3890499190 conn(0x55e07f855960 msgr2=0x55e07f855d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.563+0000 7fc00cc00640 1 -- 172.21.15.99:0/3538723476 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc000044070 con 0x7fc010101f70 2024-03-23T23:45:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.563+0000 7fc0199ec740 1 -- 172.21.15.99:0/3890499190 shutdown_connections 2024-03-23T23:45:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.563+0000 7fc0199ec740 1 -- 172.21.15.99:0/3890499190 wait complete. 2024-03-23T23:45:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.564+0000 7fc0199ec740 1 -- start start 2024-03-23T23:45:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.564+0000 7fc0199ec740 1 Processor -- start 2024-03-23T23:45:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.565+0000 7fc00cc00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 0x7fbff40a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 0x55e07f8757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc00cc00640 1 -- 172.21.15.99:0/3538723476 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0000f05b0 con 0x7fc010101f70 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc014c00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 0x7fbff40a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f87ad30 0x55e07f87d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f77ba20 con 0x55e07f87ad30 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f7bb8d0 con 0x55e07f85a3f0 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e07f873f10 con 0x55e07f85ff50 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc0199ec740 10 client.0 _ll_register_callbacks cb 0x55e07f7b81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc00f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc00f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:53894/0 (socket says 172.21.15.99:53894) 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.566+0000 7fc00f400640 1 -- 172.21.15.99:0/3634937828 learned_addr learned my addr 172.21.15.99:0/3634937828 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 0x55e07f8757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00f400640 1 -- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 msgr2=0x55e07f8757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00f400640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 0x55e07f8757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00f400640 1 -- 172.21.15.99:0/3634937828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f87ad30 msgr2=0x55e07f87d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00f400640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f87ad30 0x55e07f87d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00f400640 1 -- 172.21.15.99:0/3634937828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e07f4d4210 con 0x55e07f85ff50 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.567+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 0x55e07f8757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fc00f400640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc014d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b360e6b072ad49 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fc014c00640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 0x7fbff40a4d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55e07f85b640 tx=0x7fc008011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fbff9600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc017040 con 0x55e07f85ff50 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbffc012bd0 con 0x55e07f85ff50 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbffc013950 con 0x55e07f85ff50 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbffc009a00 con 0x55e07f85ff50 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fc0199ec740 10 client.25819 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.568+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e07f5a0620 con 0x55e07f85ff50 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbffc013950 con 0x55e07f85ff50 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbffc023070 con 0x55e07f85ff50 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 10 client.25819 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 20 client.25819 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f889ee0 0x55e07f88c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 4 client.25819 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e07f7ccd70 con 0x55e07f889ee0 2024-03-23T23:45:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.569+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f889ee0 0x55e07f88c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.571+0000 7fc00fe00640 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f889ee0 0x55e07f88c2d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1863cbfc1e3b4583 server_cookie=a3552695c95e5c71 in_seq=0 out_seq=0 2024-03-23T23:45:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.571+0000 7fbff9600640 10 client.25819 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.572+0000 7fbff9600640 1 -- 172.21.15.99:0/3634937828 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e07f7ccd70 con 0x55e07f889ee0 2024-03-23T23:45:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.572+0000 7fbff9600640 10 client.25819 handle_command_reply: tid=0 2024-03-23T23:45:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.572+0000 7fc0199ec740 1 client.25819 shutdown 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.572+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 msgr2=0x55e07f875d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc014d20 comp rx=0 tx=0).stop 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 shutdown_connections 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f889ee0 0x55e07f88c2d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f87ad30 0x55e07f87d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f85ff50 0x55e07f875d00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 --2- 172.21.15.99:0/3634937828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85a3f0 0x55e07f8757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 >> 172.21.15.99:0/3634937828 conn(0x55e07f855960 msgr2=0x55e07f854b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.573+0000 7fbff9600640 0 client.25819 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.574+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 shutdown_connections 2024-03-23T23:45:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.574+0000 7fc0199ec740 1 -- 172.21.15.99:0/3634937828 wait complete. 2024-03-23T23:45:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.575+0000 7fc0199ec740 20 client.25819 trim_cache size 0 max 16384 2024-03-23T23:45:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.604+0000 7fc0199ec740 1 Processor -- start 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.604+0000 7fc0199ec740 1 -- start start 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f859100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f874340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000e3c0 con 0x55e07f85a3f0 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000fd40 con 0x55e07f85ff50 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.605+0000 7fc0199ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000ea20 con 0x55e07f87ad30 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.606+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f859100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.606+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.606+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f859100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38730/0 (socket says 172.21.15.99:38730) 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.606+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42398/0 (socket says 172.21.15.99:42398) 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.606+0000 7fc00fe00640 1 -- 172.21.15.99:0/2997064734 learned_addr learned my addr 172.21.15.99:0/2997064734 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 -- 172.21.15.99:0/2997064734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 msgr2=0x55e07f874340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T23:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f874340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 -- 172.21.15.99:0/2997064734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 msgr2=0x55e07f859100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f859100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 -- 172.21.15.99:0/2997064734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e07f88c640 con 0x55e07f85a3f0 2024-03-23T23:45:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.607+0000 7fc00fe00640 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b98b0 tx=0x7fc000009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ee853b5b22bc7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.608+0000 7fbffbe00640 1 -- 172.21.15.99:0/2997064734 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00004b040 con 0x55e07f85a3f0 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.608+0000 7fbffbe00640 1 -- 172.21.15.99:0/2997064734 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0000bd630 con 0x55e07f85a3f0 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.608+0000 7fbffbe00640 1 -- 172.21.15.99:0/2997064734 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000bb230 con 0x55e07f85a3f0 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.609+0000 7fc0199ec740 1 -- 172.21.15.99:0/2997064734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 msgr2=0x55e07f858bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.609+0000 7fc0199ec740 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b98b0 tx=0x7fc000009260 comp rx=0 tx=0).stop 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.609+0000 7fc0199ec740 1 -- 172.21.15.99:0/2997064734 shutdown_connections 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f874340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f859100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 --2- 172.21.15.99:0/2997064734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f858bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 -- 172.21.15.99:0/2997064734 >> 172.21.15.99:0/2997064734 conn(0x55e07f855960 msgr2=0x55e07f855d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 -- 172.21.15.99:0/2997064734 shutdown_connections 2024-03-23T23:45:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.610+0000 7fc0199ec740 1 -- 172.21.15.99:0/2997064734 wait complete. 2024-03-23T23:45:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.611+0000 7fc0199ec740 1 -- start start 2024-03-23T23:45:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.611+0000 7fc0199ec740 1 Processor -- start 2024-03-23T23:45:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.612+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f89be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.612+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.612+0000 7fc0199ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f89a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38736/0 (socket says 172.21.15.99:38736) 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 -- 172.21.15.99:0/1373018857 learned_addr learned my addr 172.21.15.99:0/1373018857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000f6e0 con 0x55e07f85a3f0 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000f080 con 0x55e07f85ff50 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000c580 con 0x55e07f87ad30 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc0199ec740 10 client.0 _ll_register_callbacks cb 0x7fbffc009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc014c00640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f89a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00fe00640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f89be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 -- 172.21.15.99:0/1373018857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 msgr2=0x55e07f89a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f89a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 -- 172.21.15.99:0/1373018857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 msgr2=0x55e07f89be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f89be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00f400640 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e07f89acc0 con 0x55e07f85ff50 2024-03-23T23:45:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.613+0000 7fc00fe00640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f89be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fc00f400640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d30 tx=0x7fbffc0149c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44728506632dd28a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fbff9600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc017040 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbffc012de0 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0000bd630 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbffc03e050 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fc0199ec740 10 client.25775 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.614+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e07f8e5720 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0000bd630 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fbffc023050 con 0x55e07f85ff50 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 10 client.25775 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 20 client.25775 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f8e6020 0x55e07f8e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 4 client.25775 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e07f7c70c0 con 0x55e07f8e6020 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.615+0000 7fc00fe00640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f8e6020 0x55e07f8e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.616+0000 7fc00fe00640 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f8e6020 0x55e07f8e8440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d0a2162f42222d server_cookie=903b81b32494763b in_seq=0 out_seq=0 2024-03-23T23:45:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.616+0000 7fbff9600640 10 client.25775 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:10.621 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.620+0000 7fbff9600640 1 -- 172.21.15.99:0/1373018857 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e07f7c70c0 con 0x55e07f8e6020 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.620+0000 7fbff9600640 10 client.25775 handle_command_reply: tid=0 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.620+0000 7fc0199ec740 1 client.25775 shutdown 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 msgr2=0x55e07f89c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d30 tx=0x7fbffc0149c0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 shutdown_connections 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e07f8e6020 0x55e07f8e8440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e07f87ad30 0x55e07f89a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e07f85ff50 0x55e07f89c370 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 --2- 172.21.15.99:0/1373018857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e07f85a3f0 0x55e07f89be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 >> 172.21.15.99:0/1373018857 conn(0x55e07f855960 msgr2=0x55e07f858bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fbff9600640 0 client.25775 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 shutdown_connections 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.621+0000 7fc0199ec740 1 -- 172.21.15.99:0/1373018857 wait complete. 2024-03-23T23:45:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.622+0000 7fc0199ec740 20 client.25775 trim_cache size 0 max 16384 2024-03-23T23:45:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:45:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:45:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:45:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.622+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 msgr2=0x7fbff40a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 0x7fbff40a4d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55e07f85b640 tx=0x7fc008011920 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 msgr2=0x7fc0101018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc00002c800 tx=0x7fc000005d90 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 shutdown_connections 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbff40a2850 0x7fbff40a4d10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0101071d0 0x7fc010100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0101048a0 0x7fc0100fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 --2- 172.21.15.99:0/3538723476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc010101f70 0x7fc0101018e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 >> 172.21.15.99:0/3538723476 conn(0x7fc0100fb980 msgr2=0x7fc010102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 shutdown_connections 2024-03-23T23:45:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:10.623+0000 7fc015600640 1 -- 172.21.15.99:0/3538723476 wait complete. 2024-03-23T23:45:12.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:11 smithi099 ceph-mon[31715]: pgmap v439: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-23T23:45:12.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:11 smithi149 ceph-mon[36083]: pgmap v439: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-23T23:45:12.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:11 smithi174 ceph-mon[37002]: pgmap v439: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-23T23:45:14.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:13 smithi099 ceph-mon[31715]: pgmap v440: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-23T23:45:14.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:13 smithi149 ceph-mon[36083]: pgmap v440: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-23T23:45:14.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:13 smithi174 ceph-mon[37002]: pgmap v440: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-23T23:45:16.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:15 smithi099 ceph-mon[31715]: pgmap v441: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:45:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:15 smithi149 ceph-mon[36083]: pgmap v441: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:45:16.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:15 smithi174 ceph-mon[37002]: pgmap v441: 97 pgs: 97 active+clean; 674 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:45:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:17 smithi099 ceph-mon[31715]: pgmap v442: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:18.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:17 smithi149 ceph-mon[36083]: pgmap v442: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:18.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:17 smithi174 ceph-mon[37002]: pgmap v442: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:20.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:19 smithi099 ceph-mon[31715]: pgmap v443: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:19 smithi149 ceph-mon[36083]: pgmap v443: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:20.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:19 smithi174 ceph-mon[37002]: pgmap v443: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:21 smithi099 ceph-mon[31715]: pgmap v444: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:45:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:21 smithi149 ceph-mon[36083]: pgmap v444: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:45:22.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:21 smithi174 ceph-mon[37002]: pgmap v444: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-23T23:45:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 -- 172.21.15.99:0/4156749713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8103900 msgr2=0x7fb7d8103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 --2- 172.21.15.99:0/4156749713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8103900 0x7fb7d8103d20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009900 tx=0x7fb7c802bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 -- 172.21.15.99:0/4156749713 shutdown_connections 2024-03-23T23:45:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 --2- 172.21.15.99:0/4156749713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 0x7fb7d810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 --2- 172.21.15.99:0/4156749713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8104af0 0x7fb7d8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 --2- 172.21.15.99:0/4156749713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8103900 0x7fb7d8103d20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 -- 172.21.15.99:0/4156749713 >> 172.21.15.99:0/4156749713 conn(0x7fb7d80fee90 msgr2=0x7fb7d81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 -- 172.21.15.99:0/4156749713 shutdown_connections 2024-03-23T23:45:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.792+0000 7fb7df200640 1 -- 172.21.15.99:0/4156749713 wait complete. 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.793+0000 7fb7df200640 1 Processor -- start 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.793+0000 7fb7df200640 1 -- start start 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.793+0000 7fb7df200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8103900 0x7fb7d8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.793+0000 7fb7df200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7df200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 0x7fb7d81a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7df200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d810f5a0 con 0x7fb7d8103900 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7df200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d810f420 con 0x7fb7d81054b0 2024-03-23T23:45:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7df200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d810f720 con 0x7fb7d8104af0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8103900 0x7fb7d8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50360/0 (socket says 172.21.15.99:50360) 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 -- 172.21.15.99:0/2476168069 learned_addr learned my addr 172.21.15.99:0/2476168069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7dd200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 0x7fb7d81a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 -- 172.21.15.99:0/2476168069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 msgr2=0x7fb7d81a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 0x7fb7d81a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 -- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8103900 msgr2=0x7fb7d8199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8103900 0x7fb7d8199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.794+0000 7fb7cfe00640 1 -- 172.21.15.99:0/2476168069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d81a17a0 con 0x7fb7d8104af0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000e9f0 tx=0x7fb7c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7d5600640 1 -- 172.21.15.99:0/2476168069 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000cd80 con 0x7fb7d8104af0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7d5600640 1 -- 172.21.15.99:0/2476168069 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7c0010910 con 0x7fb7d8104af0 2024-03-23T23:45:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d81a1a90 con 0x7fb7d8104af0 2024-03-23T23:45:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7d5600640 1 -- 172.21.15.99:0/2476168069 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0018990 con 0x7fb7d8104af0 2024-03-23T23:45:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.795+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d81a2030 con 0x7fb7d8104af0 2024-03-23T23:45:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 Processor -- start 2024-03-23T23:45:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 -- start start 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d3066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d2c43b0 con 0x55e88d3011e0 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d266830 con 0x55e88d301bd0 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d20f5d0 con 0x55e88d306d40 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d3066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43562/0 (socket says 172.21.15.99:43562) 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.799+0000 7fb7dd200640 1 -- 172.21.15.99:0/1411412613 learned_addr learned my addr 172.21.15.99:0/1411412613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7d5600640 1 -- 172.21.15.99:0/2476168069 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb7c0010ac0 con 0x7fb7d8104af0 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 -- 172.21.15.99:0/1411412613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 msgr2=0x55e88d3091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 -- 172.21.15.99:0/1411412613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 msgr2=0x55e88d3066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d3066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 -- 172.21.15.99:0/1411412613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e88d2272c0 con 0x55e88d3011e0 2024-03-23T23:45:23.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d3066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.800+0000 7fb7dd200640 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009fd0 tx=0x7fb7d00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eb84d4830e85c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.801+0000 7fb7cd600640 1 -- 172.21.15.99:0/1411412613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0004d20 con 0x55e88d3011e0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.801+0000 7fb7cd600640 1 -- 172.21.15.99:0/1411412613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7d0004050 con 0x55e88d3011e0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.801+0000 7fb7cd600640 1 -- 172.21.15.99:0/1411412613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0005260 con 0x55e88d3011e0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.801+0000 7fb7d5600640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 0x7fb7bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7d5600640 1 -- 172.21.15.99:0/2476168069 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb7c00ced10 con 0x7fb7d8104af0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 0x7fb7bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 -- 172.21.15.99:0/1411412613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 msgr2=0x55e88d301600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009fd0 tx=0x7fb7d00118d0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 -- 172.21.15.99:0/1411412613 shutdown_connections 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d3066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/1411412613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d301600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 -- 172.21.15.99:0/1411412613 >> 172.21.15.99:0/1411412613 conn(0x55e88d2fc750 msgr2=0x55e88d2fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 -- 172.21.15.99:0/1411412613 shutdown_connections 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.802+0000 7fb7e1e26740 1 -- 172.21.15.99:0/1411412613 wait complete. 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.803+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 0x7fb7bc0a4de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7c802c170 tx=0x7fb7c802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.803+0000 7fb7e1e26740 1 -- start start 2024-03-23T23:45:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.803+0000 7fb7e1e26740 1 Processor -- start 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d30b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d222a20 con 0x55e88d3011e0 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d2628d0 con 0x55e88d301bd0 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88d309cf0 con 0x55e88d306d40 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7e1e26740 10 client.0 _ll_register_callbacks cb 0x55e88d25f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d30b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43572/0 (socket says 172.21.15.99:43572) 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 -- 172.21.15.99:0/4142888471 learned_addr learned my addr 172.21.15.99:0/4142888471 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50378/0 (socket says 172.21.15.99:50378) 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 -- 172.21.15.99:0/4142888471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 msgr2=0x55e88d3129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 -- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 msgr2=0x55e88d30b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d30b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7d7e00640 1 -- 172.21.15.99:0/4142888471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e88cf7b210 con 0x55e88d3011e0 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.804+0000 7fb7dd200640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8035000 tx=0x7fb7c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405382d1d4164da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:45:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c804f070 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7d0004050 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb7c8003800 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c8003a00 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7e1e26740 10 client.15624 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.805+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e88d047620 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7d0004050 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb7c8034440 con 0x55e88d3011e0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 20 client.15624 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d3172b0 0x55e88d3196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 4 client.15624 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.806+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e88d273d70 con 0x55e88d3172b0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.807+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d3172b0 0x55e88d3196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.807+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d3172b0 0x55e88d3196a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41d29558c94cbf32 server_cookie=d408aacd39ebf1e3 in_seq=0 out_seq=0 2024-03-23T23:45:23.810 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.807+0000 7fb7c6000640 10 client.15624 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7c6000640 1 -- 172.21.15.99:0/4142888471 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e88d273d70 con 0x55e88d3172b0 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7c6000640 10 client.15624 handle_command_reply: tid=0 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 client.15624 shutdown 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 msgr2=0x55e88d30b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8035000 tx=0x7fb7c802bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 shutdown_connections 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d3172b0 0x55e88d3196a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d306d40 0x55e88d3129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d301bd0 0x55e88d30b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/4142888471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d30b450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 >> 172.21.15.99:0/4142888471 conn(0x55e88d2fc750 msgr2=0x55e88d2fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.808+0000 7fb7c6000640 0 client.15624 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.809+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 shutdown_connections 2024-03-23T23:45:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.809+0000 7fb7e1e26740 1 -- 172.21.15.99:0/4142888471 wait complete. 2024-03-23T23:45:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.809+0000 7fb7e1e26740 20 client.15624 trim_cache size 0 max 16384 2024-03-23T23:45:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 Processor -- start 2024-03-23T23:45:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 -- start start 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 0x55e88d302460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 0x55e88d30b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81aac10 con 0x55e88d306d40 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81ac590 con 0x55e88d3011e0 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.836+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81ab270 con 0x55e88d301bd0 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 0x55e88d302460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50390/0 (socket says 172.21.15.99:50390) 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 -- 172.21.15.99:0/3562259438 learned_addr learned my addr 172.21.15.99:0/3562259438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 0x55e88d302460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:58304/0 (socket says 172.21.15.99:58304) 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7dd200640 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 0x55e88d30b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 -- 172.21.15.99:0/3562259438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 msgr2=0x55e88d302460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 0x55e88d302460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 -- 172.21.15.99:0/3562259438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 msgr2=0x55e88d30b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 0x55e88d30b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 -- 172.21.15.99:0/3562259438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e88d309e30 con 0x55e88d301bd0 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7dd200640 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 0x55e88d30b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:23.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0098110 tx=0x7fb7c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d42281e8fcf32b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7ccc00640 1 -- 172.21.15.99:0/3562259438 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00a3e30 con 0x55e88d301bd0 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7ccc00640 1 -- 172.21.15.99:0/3562259438 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7c00b4070 con 0x55e88d301bd0 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.837+0000 7fb7ccc00640 1 -- 172.21.15.99:0/3562259438 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0018d90 con 0x55e88d301bd0 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 -- 172.21.15.99:0/3562259438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 msgr2=0x55e88d312990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0098110 tx=0x7fb7c0000bc0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 -- 172.21.15.99:0/3562259438 shutdown_connections 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d306d40 0x55e88d30b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d312990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/3562259438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d3011e0 0x55e88d302460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 -- 172.21.15.99:0/3562259438 >> 172.21.15.99:0/3562259438 conn(0x55e88d2fc750 msgr2=0x55e88d30b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 -- 172.21.15.99:0/3562259438 shutdown_connections 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.838+0000 7fb7e1e26740 1 -- 172.21.15.99:0/3562259438 wait complete. 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 -- start start 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 Processor -- start 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d310560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d306d40 0x55e88d311ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81abf30 con 0x55e88d3011e0 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81ab8d0 con 0x55e88d306d40 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81a8dd0 con 0x55e88d301bd0 2024-03-23T23:45:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.839+0000 7fb7e1e26740 10 client.0 _ll_register_callbacks cb 0x7fb7c8043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50398/0 (socket says 172.21.15.99:50398) 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 -- 172.21.15.99:0/269449132 learned_addr learned my addr 172.21.15.99:0/269449132 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d310560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7dd200640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d306d40 0x55e88d311ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 -- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d306d40 msgr2=0x55e88d311ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d306d40 0x55e88d311ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 -- 172.21.15.99:0/269449132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 msgr2=0x55e88d310560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d310560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 -- 172.21.15.99:0/269449132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e88d3120a0 con 0x55e88d301bd0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7cfe00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00a3900 tx=0x7fb7c00cc900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7f9d898a835251 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00ae020 con 0x55e88d301bd0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb7c00b4220 con 0x55e88d301bd0 2024-03-23T23:45:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.840+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c00b4070 con 0x55e88d301bd0 2024-03-23T23:45:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.841+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c00a7050 con 0x55e88d301bd0 2024-03-23T23:45:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.841+0000 7fb7e1e26740 10 client.25837 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.841+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e88d37b750 con 0x55e88d301bd0 2024-03-23T23:45:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7c00b4070 con 0x55e88d301bd0 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb7c009ab90 con 0x55e88d301bd0 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 10 client.25837 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 20 client.25837 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d37c020 0x55e88d37e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 4 client.25837 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.842+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e88d26e0c0 con 0x55e88d37c020 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.843+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d37c020 0x55e88d37e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.843+0000 7fb7c6000640 10 client.25837 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.843+0000 7fb7d7e00640 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d37c020 0x55e88d37e440 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8251f3a89b61f96d server_cookie=bace7d3f29ca7faa in_seq=0 out_seq=0 2024-03-23T23:45:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.846+0000 7fb7c6000640 1 -- 172.21.15.99:0/269449132 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e88d26e0c0 con 0x55e88d37c020 2024-03-23T23:45:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.846+0000 7fb7c6000640 10 client.25837 handle_command_reply: tid=0 2024-03-23T23:45:23.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 client.25837 shutdown 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 msgr2=0x55e88d310aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00a3900 tx=0x7fb7c00cc900 comp rx=0 tx=0).stop 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 shutdown_connections 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55e88d37c020 0x55e88d37e440 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e88d306d40 0x55e88d311ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e88d301bd0 0x55e88d310aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 --2- 172.21.15.99:0/269449132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e88d3011e0 0x55e88d310560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7c6000640 0 client.25837 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 >> 172.21.15.99:0/269449132 conn(0x55e88d2fc750 msgr2=0x55e88d2ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 shutdown_connections 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 1 -- 172.21.15.99:0/269449132 wait complete. 2024-03-23T23:45:23.848 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.847+0000 7fb7e1e26740 20 client.25837 trim_cache size 0 max 16384 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 msgr2=0x7fb7bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 0x7fb7bc0a4de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7c802c170 tx=0x7fb7c802df90 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 msgr2=0x7fb7d819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000e9f0 tx=0x7fb7c000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 shutdown_connections 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb7bc0a2920 0x7fb7bc0a4de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7d81054b0 0x7fb7d81a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d8104af0 0x7fb7d819a0c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 --2- 172.21.15.99:0/2476168069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7d8103900 0x7fb7d8199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 >> 172.21.15.99:0/2476168069 conn(0x7fb7d80fee90 msgr2=0x7fb7d8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 shutdown_connections 2024-03-23T23:45:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:23.848+0000 7fb7df200640 1 -- 172.21.15.99:0/2476168069 wait complete. 2024-03-23T23:45:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:23 smithi099 ceph-mon[31715]: pgmap v445: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T23:45:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:23 smithi149 ceph-mon[36083]: pgmap v445: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T23:45:24.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:23 smithi174 ceph-mon[37002]: pgmap v445: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-23T23:45:26.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:25 smithi099 ceph-mon[31715]: pgmap v446: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:25 smithi149 ceph-mon[36083]: pgmap v446: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:26.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:26 smithi174 ceph-mon[37002]: pgmap v446: 97 pgs: 97 active+clean; 675 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-23T23:45:27.274 INFO:tasks.workunit.client.0.smithi099.stderr:Updating files: 54% (6684/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 91% (11202/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.692 INFO:tasks.workunit.client.0.smithi099.stderr: git switch -c 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr:Or undo this operation with: 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr: git switch - 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-23T23:45:27.693 INFO:tasks.workunit.client.0.smithi099.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-23T23:45:27.701 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-23T23:45:27.758 INFO:tasks.workunit.client.0.smithi099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T23:45:27.765 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T23:45:27.765 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T23:45:28.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:28 smithi099 ceph-mon[31715]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-23T23:45:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:28 smithi149 ceph-mon[36083]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-23T23:45:28.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:28 smithi174 ceph-mon[37002]: pgmap v447: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-23T23:45:28.696 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T23:45:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T23:45:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T23:45:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T23:45:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T23:45:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T23:45:29.026 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:45:29.026 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-23T23:45:29.066 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-03-23T23:45:29.067 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-03-23T23:45:29.067 DEBUG:teuthology.orchestra.run.smithi099:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-03-23T23:45:29.173 INFO:tasks.workunit.client.0.smithi099.stdout:test large (16MB) dio write 2024-03-23T23:45:29.173 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo 'test large (16MB) dio write' 2024-03-23T23:45:29.174 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-03-23T23:45:29.401 INFO:tasks.workunit.client.0.smithi099.stderr:1+0 records in 2024-03-23T23:45:29.401 INFO:tasks.workunit.client.0.smithi099.stderr:1+0 records out 2024-03-23T23:45:29.401 INFO:tasks.workunit.client.0.smithi099.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.222695 s, 75.3 MB/s 2024-03-23T23:45:29.404 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo OK 2024-03-23T23:45:29.404 INFO:tasks.workunit.client.0.smithi099.stdout:OK 2024-03-23T23:45:29.405 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T23:45:29.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T23:45:29.500 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-03-23T23:45:29.500 DEBUG:teuthology.orchestra.run.smithi099:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-03-23T23:45:30.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:45:30.068 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:45:30.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 -- 172.21.15.99:0/3131118284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb81016b0 msgr2=0x7f1fb8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 --2- 172.21.15.99:0/3131118284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb81016b0 0x7f1fb8101ad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02bc50 comp rx=0 tx=0).stop 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 -- 172.21.15.99:0/3131118284 shutdown_connections 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 --2- 172.21.15.99:0/3131118284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb8103260 0x7f1fb8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 --2- 172.21.15.99:0/3131118284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81028a0 0x7f1fb8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 --2- 172.21.15.99:0/3131118284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb81016b0 0x7f1fb8101ad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 -- 172.21.15.99:0/3131118284 >> 172.21.15.99:0/3131118284 conn(0x7f1fb80fcc40 msgr2=0x7f1fb80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 -- 172.21.15.99:0/3131118284 shutdown_connections 2024-03-23T23:45:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.282+0000 7f1fbf400640 1 -- 172.21.15.99:0/3131118284 wait complete. 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.284+0000 7f1fbf400640 1 Processor -- start 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.284+0000 7f1fbf400640 1 -- start start 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 0x7f1fb8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb8103260 0x7f1fb819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810d370 con 0x7f1fb81028a0 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810d1f0 con 0x7f1fb8103260 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbf400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810d4f0 con 0x7f1fb81016b0 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40586/0 (socket says 172.21.15.99:40586) 2024-03-23T23:45:30.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 -- 172.21.15.99:0/1648702612 learned_addr learned my addr 172.21.15.99:0/1648702612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbd400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb8103260 0x7f1fb819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 0x7f1fb8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 -- 172.21.15.99:0/1648702612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb8103260 msgr2=0x7f1fb819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb8103260 0x7f1fb819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 -- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 msgr2=0x7f1fb8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 0x7f1fb8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fbca00640 1 -- 172.21.15.99:0/1648702612 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb819f670 con 0x7f1fb81016b0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.285+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 0x7f1fb8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fbca00640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005ab0 tx=0x7f1fac02c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fb5600640 1 -- 172.21.15.99:0/1648702612 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac041070 con 0x7f1fb81016b0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fb5600640 1 -- 172.21.15.99:0/1648702612 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fac004450 con 0x7f1fb81016b0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb80738d0 con 0x7f1fb81016b0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fb5600640 1 -- 172.21.15.99:0/1648702612 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac034440 con 0x7f1fb81016b0 2024-03-23T23:45:30.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.286+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb8073e20 con 0x7f1fb81016b0 2024-03-23T23:45:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 Processor -- start 2024-03-23T23:45:30.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 -- start start 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aeeda250 con 0x55a3aef75870 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aee82a30 con 0x55a3aef7a9e0 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.290+0000 7f1fc21df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aeed62f0 con 0x55a3aef74e80 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37856/0 (socket says 172.21.15.99:37856) 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fb7e00640 1 -- 172.21.15.99:0/2278503346 learned_addr learned my addr 172.21.15.99:0/2278503346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 -- 172.21.15.99:0/2278503346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 msgr2=0x55a3aef7ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 -- 172.21.15.99:0/2278503346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 msgr2=0x55a3aef7a370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7a370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 -- 172.21.15.99:0/2278503346 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3aee6d690 con 0x55a3aef74e80 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fb5600640 1 -- 172.21.15.99:0/1648702612 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1fac044070 con 0x7f1fb81016b0 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.291+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef752a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009f50 tx=0x7f1fb0003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5ebf19672d73483 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.292+0000 7f1fa9c00640 1 -- 172.21.15.99:0/2278503346 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0011440 con 0x55a3aef74e80 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.292+0000 7f1fa9c00640 1 -- 172.21.15.99:0/2278503346 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0015030 con 0x55a3aef74e80 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.292+0000 7f1fa9c00640 1 -- 172.21.15.99:0/2278503346 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010a00 con 0x55a3aef74e80 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.292+0000 7f1fc21df740 1 -- 172.21.15.99:0/2278503346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 msgr2=0x55a3aef752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.292+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef752a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009f50 tx=0x7f1fb0003e00 comp rx=0 tx=0).stop 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 -- 172.21.15.99:0/2278503346 shutdown_connections 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2278503346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef752a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 -- 172.21.15.99:0/2278503346 >> 172.21.15.99:0/2278503346 conn(0x55a3aef3d200 msgr2=0x55a3aef707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 -- 172.21.15.99:0/2278503346 shutdown_connections 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 -- 172.21.15.99:0/2278503346 wait complete. 2024-03-23T23:45:30.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 -- start start 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.293+0000 7f1fc21df740 1 Processor -- start 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aef7d990 con 0x55a3aef75870 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aee95e80 con 0x55a3aef7a9e0 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aef7db10 con 0x55a3aef74e80 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fb5600640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 0x7f1fa00a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fc21df740 10 client.0 _ll_register_callbacks cb 0x55a3aeea8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.294+0000 7f1fb5600640 1 -- 172.21.15.99:0/1648702612 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1fac0f0720 con 0x7f1fb81016b0 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.295+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40614/0 (socket says 172.21.15.99:40614) 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.295+0000 7f1fbd400640 1 -- 172.21.15.99:0/2788114023 learned_addr learned my addr 172.21.15.99:0/2788114023 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.295+0000 7f1fbd400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 0x7f1fa00a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.295+0000 7f1fbca00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.295+0000 7f1fbd400640 1 -- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 msgr2=0x55a3aef86470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef86470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 -- 172.21.15.99:0/2788114023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 msgr2=0x55a3aef7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 -- 172.21.15.99:0/2788114023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3aecb87d0 con 0x55a3aef74e80 2024-03-23T23:45:30.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbca00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef86470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0011010 tx=0x7f1fb00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30e62ab7ea8fc85 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.296+0000 7f1fbd400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 0x7f1fa00a4bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0004410 tx=0x7f1fb001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb001e070 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1fb0004a40 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fb0015030 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb0039050 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 10 client.25849 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3aebee3f0 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0015030 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1fb00056c0 con 0x55a3aef74e80 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 10 client.25849 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:30.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 20 client.25849 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.297+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aef932e0 0x55a3aef956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.298+0000 7f1fc21df740 4 client.25849 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.298+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3aeee7790 con 0x55a3aef932e0 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.298+0000 7f1fbca00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aef932e0 0x55a3aef956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.298+0000 7f1fbca00640 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aef932e0 0x55a3aef956d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4719b7b636728c23 server_cookie=fd98f59c93ea2ca6 in_seq=0 out_seq=0 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.298+0000 7f1f9e000640 10 client.25849 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1f9e000640 1 -- 172.21.15.99:0/2788114023 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a3aeee7790 con 0x55a3aef932e0 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1f9e000640 10 client.25849 handle_command_reply: tid=0 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 client.25849 shutdown 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 msgr2=0x55a3aef7eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0011010 tx=0x7f1fb00107a0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 shutdown_connections 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aef932e0 0x55a3aef956d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef7a9e0 0x55a3aef86470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef75870 0x55a3aef7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 --2- 172.21.15.99:0/2788114023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef74e80 0x55a3aef7eef0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 >> 172.21.15.99:0/2788114023 conn(0x55a3aef3d200 msgr2=0x55a3aef6fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.299+0000 7f1f9e000640 0 client.25849 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.300+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 shutdown_connections 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.300+0000 7f1fc21df740 1 -- 172.21.15.99:0/2788114023 wait complete. 2024-03-23T23:45:30.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.300+0000 7f1fc21df740 20 client.25849 trim_cache size 0 max 16384 2024-03-23T23:45:30.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.317+0000 7f1fc21df740 1 Processor -- start 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.317+0000 7f1fc21df740 1 -- start start 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 0x55a3aef7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef7a9e0 0x55a3aef76a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a6870 con 0x55a3aef7a9e0 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a81f0 con 0x55a3aef74e80 2024-03-23T23:45:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fc21df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a6ed0 con 0x55a3aef75870 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fbca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 0x55a3aef7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40616/0 (socket says 172.21.15.99:40616) 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 -- 172.21.15.99:0/1332648265 learned_addr learned my addr 172.21.15.99:0/1332648265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fbd400640 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef7a9e0 0x55a3aef76a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 -- 172.21.15.99:0/1332648265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 msgr2=0x55a3aef7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 0x55a3aef7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 -- 172.21.15.99:0/1332648265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef7a9e0 msgr2=0x55a3aef76a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef7a9e0 0x55a3aef76a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fb7e00640 1 -- 172.21.15.99:0/1332648265 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3aebae0f0 con 0x55a3aef75870 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.318+0000 7f1fbca00640 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 0x55a3aef7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.319+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4013bc0 tx=0x7f1fa4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd380bf10059514 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.319+0000 7f1fa9200640 1 -- 172.21.15.99:0/1332648265 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40130d0 con 0x55a3aef75870 2024-03-23T23:45:30.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.319+0000 7f1fa9200640 1 -- 172.21.15.99:0/1332648265 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa4013270 con 0x55a3aef75870 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.319+0000 7f1fa9200640 1 -- 172.21.15.99:0/1332648265 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4011440 con 0x55a3aef75870 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- 172.21.15.99:0/1332648265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 msgr2=0x55a3aef86400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4013bc0 tx=0x7f1fa4005f90 comp rx=0 tx=0).stop 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- 172.21.15.99:0/1332648265 shutdown_connections 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef7a9e0 0x55a3aef76a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef75870 0x55a3aef86400 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 --2- 172.21.15.99:0/1332648265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef74e80 0x55a3aef7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- 172.21.15.99:0/1332648265 >> 172.21.15.99:0/1332648265 conn(0x55a3aef3d200 msgr2=0x55a3aef76310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- 172.21.15.99:0/1332648265 shutdown_connections 2024-03-23T23:45:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- 172.21.15.99:0/1332648265 wait complete. 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 -- start start 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.320+0000 7f1fc21df740 1 Processor -- start 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 0x55a3aef83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef75870 0x55a3aef83ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a7b90 con 0x55a3aef74e80 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a7530 con 0x55a3aef75870 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81a4a30 con 0x55a3aef7a9e0 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fc21df740 10 client.0 _ll_register_callbacks cb 0x7f1fa4013850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40620/0 (socket says 172.21.15.99:40620) 2024-03-23T23:45:30.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 -- 172.21.15.99:0/245554218 learned_addr learned my addr 172.21.15.99:0/245554218 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fb7e00640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef75870 0x55a3aef83ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbca00640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 0x55a3aef83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 -- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef75870 msgr2=0x55a3aef83ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef75870 0x55a3aef83ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 -- 172.21.15.99:0/245554218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 msgr2=0x55a3aef83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 0x55a3aef83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.321+0000 7f1fbd400640 1 -- 172.21.15.99:0/245554218 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3aef852a0 con 0x55a3aef7a9e0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1fbca00640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 0x55a3aef83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1fbd400640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000f80 tx=0x7f1fb0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9176d0c1d96083a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb001e040 con 0x55a3aef7a9e0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1fb001a990 con 0x55a3aef7a9e0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fa4013270 con 0x55a3aef7a9e0 2024-03-23T23:45:30.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb0024040 con 0x55a3aef7a9e0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1fc21df740 10 client.25855 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.322+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a3aef86620 con 0x55a3aef7a9e0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa4013270 con 0x55a3aef7a9e0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1fb001acb0 con 0x55a3aef7a9e0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 10 client.25855 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 20 client.25855 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aefef050 0x55a3aeff1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 4 client.25855 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3aeee1ae0 con 0x55a3aefef050 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.323+0000 7f1fbca00640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aefef050 0x55a3aeff1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1f9e000640 10 client.25855 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fbca00640 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aefef050 0x55a3aeff1470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06ba6a4956821cc server_cookie=279841c65bac848 in_seq=0 out_seq=0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1f9e000640 1 -- 172.21.15.99:0/245554218 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a3aeee1ae0 con 0x55a3aefef050 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1f9e000640 10 client.25855 handle_command_reply: tid=0 2024-03-23T23:45:30.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 client.25855 shutdown 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 msgr2=0x55a3aef84cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000f80 tx=0x7f1fb0005f90 comp rx=0 tx=0).stop 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 shutdown_connections 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a3aefef050 0x55a3aeff1470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3aef7a9e0 0x55a3aef84cb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3aef75870 0x55a3aef83ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 --2- 172.21.15.99:0/245554218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3aef74e80 0x55a3aef83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 >> 172.21.15.99:0/245554218 conn(0x55a3aef3d200 msgr2=0x55a3aef76ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1f9e000640 0 client.25855 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 shutdown_connections 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.324+0000 7f1fc21df740 1 -- 172.21.15.99:0/245554218 wait complete. 2024-03-23T23:45:30.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fc21df740 20 client.25855 trim_cache size 0 max 16384 2024-03-23T23:45:30.329 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:45:30.329 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-23T23:45:30.329 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-23T23:45:30.329 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 msgr2=0x7f1fa00a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 0x7f1fa00a4bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0004410 tx=0x7f1fb001b000 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 msgr2=0x7f1fb8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005ab0 tx=0x7f1fac02c6e0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 shutdown_connections 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1fa00a2710 0x7f1fa00a4bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb8103260 0x7f1fb819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb81028a0 0x7f1fb8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 --2- 172.21.15.99:0/1648702612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb81016b0 0x7f1fb8197a50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.325+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 >> 172.21.15.99:0/1648702612 conn(0x7f1fb80fcc40 msgr2=0x7f1fb8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.326+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 shutdown_connections 2024-03-23T23:45:30.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.326+0000 7f1fbf400640 1 -- 172.21.15.99:0/1648702612 wait complete. 2024-03-23T23:45:30.345 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:30 smithi099 ceph-mon[31715]: pgmap v448: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:45:30.348 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T23:45:30.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:45:30.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:30 smithi149 ceph-mon[36083]: pgmap v448: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:45:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:30 smithi174 ceph-mon[37002]: pgmap v448: 97 pgs: 97 active+clean; 676 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-23T23:45:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 -- 172.21.15.99:0/433455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc1016b0 msgr2=0x7f5fcc101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 --2- 172.21.15.99:0/433455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc101ad0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc002bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 -- 172.21.15.99:0/433455304 shutdown_connections 2024-03-23T23:45:30.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 --2- 172.21.15.99:0/433455304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc103260 0x7f5fcc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 --2- 172.21.15.99:0/433455304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 --2- 172.21.15.99:0/433455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc101ad0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 -- 172.21.15.99:0/433455304 >> 172.21.15.99:0/433455304 conn(0x7f5fcc0fcc80 msgr2=0x7f5fcc0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 -- 172.21.15.99:0/433455304 shutdown_connections 2024-03-23T23:45:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.568+0000 7f5fd3600640 1 -- 172.21.15.99:0/433455304 wait complete. 2024-03-23T23:45:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.569+0000 7f5fd3600640 1 Processor -- start 2024-03-23T23:45:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 -- start start 2024-03-23T23:45:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 0x7f5fcc073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d350 con 0x7f5fcc1016b0 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d1d0 con 0x7f5fcc103260 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd3600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d4d0 con 0x7f5fcc1028a0 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd0c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57102/0 (socket says 172.21.15.99:57102) 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd0c00640 1 -- 172.21.15.99:0/3893810665 learned_addr learned my addr 172.21.15.99:0/3893810665 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fcbe00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.570+0000 7f5fd1600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 0x7f5fcc073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 -- 172.21.15.99:0/3893810665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 msgr2=0x7f5fcc073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 -- 172.21.15.99:0/3893810665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 msgr2=0x7f5fcc073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 0x7f5fcc073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 -- 172.21.15.99:0/3893810665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc074860 con 0x7f5fcc1016b0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd1600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 0x7f5fcc073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fcbe00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fd0c00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0000c00 tx=0x7f5fc002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.571+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0043020 con 0x7f5fcc1016b0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.572+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc00054e0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.572+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0003ce0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.572+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc1a1ae0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.572+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc1a1fb0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.573+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc1089d0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.576+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5fc0046020 con 0x7f5fcc1016b0 2024-03-23T23:45:30.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.578+0000 7f5fc9600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 0x7f5fb40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.578+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5fc00f0bc0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.578+0000 7f5fcbe00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 0x7f5fb40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:30.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.579+0000 7f5fcbe00640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 0x7f5fb40a4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0751c0 tx=0x7f5fb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:30.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.580+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fc004d030 con 0x7f5fcc1016b0 2024-03-23T23:45:30.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.859+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fcc02d230 con 0x7f5fcc1016b0 2024-03-23T23:45:30.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.861+0000 7f5fc9600640 1 -- 172.21.15.99:0/3893810665 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f5fc00bdbe0 con 0x7f5fcc1016b0 2024-03-23T23:45:30.861 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:45:30.862 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:45:30.862 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:45:30.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 msgr2=0x7f5fb40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 0x7f5fb40a4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0751c0 tx=0x7f5fb8005f90 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 msgr2=0x7f5fcc075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0000c00 tx=0x7f5fc002c6c0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 shutdown_connections 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5fb40a2880 0x7f5fb40a4d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fcc103260 0x7f5fcc073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc1028a0 0x7f5fcc073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 --2- 172.21.15.99:0/3893810665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fcc1016b0 0x7f5fcc075450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 >> 172.21.15.99:0/3893810665 conn(0x7f5fcc0fcc80 msgr2=0x7f5fcc105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 shutdown_connections 2024-03-23T23:45:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:30.864+0000 7f5fd3600640 1 -- 172.21.15.99:0/3893810665 wait complete. 2024-03-23T23:45:30.884 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:45:30.884 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T23:45:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 -- 172.21.15.99:0/1665672616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0901037a0 msgr2=0x7ff090103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 --2- 172.21.15.99:0/1665672616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0901037a0 0x7ff090103bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 -- 172.21.15.99:0/1665672616 shutdown_connections 2024-03-23T23:45:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 --2- 172.21.15.99:0/1665672616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 0x7ff09010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 --2- 172.21.15.99:0/1665672616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff090104990 0x7ff090104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 --2- 172.21.15.99:0/1665672616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0901037a0 0x7ff090103bc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 -- 172.21.15.99:0/1665672616 >> 172.21.15.99:0/1665672616 conn(0x7ff0900fed10 msgr2=0x7ff090101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 -- 172.21.15.99:0/1665672616 shutdown_connections 2024-03-23T23:45:31.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.126+0000 7ff08f200640 1 -- 172.21.15.99:0/1665672616 wait complete. 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 Processor -- start 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 -- start start 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 0x7ff0901a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010f460 con 0x7ff090105350 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010f2e0 con 0x7ff090104990 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.127+0000 7ff08f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09010f5e0 con 0x7ff0901037a0 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff08de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff08de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40664/0 (socket says 172.21.15.99:40664) 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37910/0 (socket says 172.21.15.99:37910) 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff08de00640 1 -- 172.21.15.99:0/3043645530 learned_addr learned my addr 172.21.15.99:0/3043645530 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff08e800640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 0x7ff0901a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 -- 172.21.15.99:0/3043645530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 msgr2=0x7ff090199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 -- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 msgr2=0x7ff0901a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 0x7ff0901a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 -- 172.21.15.99:0/3043645530 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0901a16f0 con 0x7ff090104990 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff08de00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.128+0000 7ff087e00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9f0 tx=0x7ff07800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.129+0000 7ff086a00640 1 -- 172.21.15.99:0/3043645530 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800ccc0 con 0x7ff090104990 2024-03-23T23:45:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.129+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0901a19e0 con 0x7ff090104990 2024-03-23T23:45:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.129+0000 7ff086a00640 1 -- 172.21.15.99:0/3043645530 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff078005500 con 0x7ff090104990 2024-03-23T23:45:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.129+0000 7ff086a00640 1 -- 172.21.15.99:0/3043645530 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078010690 con 0x7ff090104990 2024-03-23T23:45:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.129+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0901a1f30 con 0x7ff090104990 2024-03-23T23:45:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.133+0000 7ff097682740 1 Processor -- start 2024-03-23T23:45:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.133+0000 7ff097682740 1 -- start start 2024-03-23T23:45:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.133+0000 7ff086a00640 1 -- 172.21.15.99:0/3043645530 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff0780046c0 con 0x7ff090104990 2024-03-23T23:45:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc7c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc82e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46dbdf2c0 con 0x55b46dc7c580 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46db87c90 con 0x55b46dc7bbc0 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff097682740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46dbdb360 con 0x55b46dc7a9f0 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40666/0 (socket says 172.21.15.99:40666) 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 -- 172.21.15.99:0/3589644026 learned_addr learned my addr 172.21.15.99:0/3589644026 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08de00640 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc7c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 -- 172.21.15.99:0/3589644026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 msgr2=0x55b46dc7c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc7c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 -- 172.21.15.99:0/3589644026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 msgr2=0x55b46dc82e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc82e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.134+0000 7ff08e800640 1 -- 172.21.15.99:0/3589644026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46dbac080 con 0x55b46dc7a9f0 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff08de00640 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc7c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff08e800640 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff088009fc0 tx=0x7ff088010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10a964bd7fbbce2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff086a00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 0x7ff0740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff07f000640 1 -- 172.21.15.99:0/3589644026 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088003b70 con 0x55b46dc7a9f0 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff07f000640 1 -- 172.21.15.99:0/3589644026 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff088003d10 con 0x55b46dc7a9f0 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff086a00640 1 -- 172.21.15.99:0/3043645530 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff0780cedd0 con 0x7ff090104990 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff07f000640 1 -- 172.21.15.99:0/3589644026 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088011aa0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.135+0000 7ff08de00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 0x7ff0740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 -- 172.21.15.99:0/3589644026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 msgr2=0x55b46dc7ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff088009fc0 tx=0x7ff088010860 comp rx=0 tx=0).stop 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 -- 172.21.15.99:0/3589644026 shutdown_connections 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc82e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc7c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 --2- 172.21.15.99:0/3589644026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ae10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 -- 172.21.15.99:0/3589644026 >> 172.21.15.99:0/3589644026 conn(0x55b46dc75de0 msgr2=0x55b46dc761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 -- 172.21.15.99:0/3589644026 shutdown_connections 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff08de00640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 0x7ff0740a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff08002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.136+0000 7ff097682740 1 -- 172.21.15.99:0/3589644026 wait complete. 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 -- start start 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 Processor -- start 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc85240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc8c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46dc83590 con 0x55b46dc7c580 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46db9b0d0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.137+0000 7ff097682740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46dc83710 con 0x55b46dc7a9f0 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff097682740 10 client.0 _ll_register_callbacks cb 0x55b46db9fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40678/0 (socket says 172.21.15.99:40678) 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 -- 172.21.15.99:0/2834668943 learned_addr learned my addr 172.21.15.99:0/2834668943 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08e800640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc8c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff087e00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc85240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 -- 172.21.15.99:0/2834668943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 msgr2=0x55b46dc85240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc85240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 -- 172.21.15.99:0/2834668943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 msgr2=0x55b46dc8c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc8c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 -- 172.21.15.99:0/2834668943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b46d8f33f0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.138+0000 7ff08de00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff080036040 tx=0x7ff080009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8b5bfb8e5c03dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff073400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080045070 con 0x55b46dc7a9f0 2024-03-23T23:45:31.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff080004bf0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff088003d10 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff080004ed0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff097682740 10 client.25873 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b46d9bd7d0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.139+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff088003d10 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff080003c20 con 0x55b46dc7a9f0 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 10 client.25873 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 10 client.25873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 20 client.25873 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dc98f80 0x55b46dc9b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 4 client.25873 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b46dbec800 con 0x55b46dc98f80 2024-03-23T23:45:31.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.140+0000 7ff087e00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dc98f80 0x55b46dc9b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.141+0000 7ff087e00640 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dc98f80 0x55b46dc9b3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b37b2e17ea4cfe8 server_cookie=db4e96571ba9ad3d in_seq=0 out_seq=0 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.141+0000 7ff073400640 10 client.25873 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.141+0000 7ff073400640 1 -- 172.21.15.99:0/2834668943 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b46dbec800 con 0x55b46dc98f80 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff073400640 10 client.25873 handle_command_reply: tid=0 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 client.25873 shutdown 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 msgr2=0x55b46dc84d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff080036040 tx=0x7ff080009260 comp rx=0 tx=0).stop 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 shutdown_connections 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dc98f80 0x55b46dc9b3a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc8c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7bbc0 0x55b46dc85240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff073400640 0 client.25873 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 --2- 172.21.15.99:0/2834668943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7a9f0 0x55b46dc84d00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 >> 172.21.15.99:0/2834668943 conn(0x55b46dc75de0 msgr2=0x55b46dc75160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 shutdown_connections 2024-03-23T23:45:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 1 -- 172.21.15.99:0/2834668943 wait complete. 2024-03-23T23:45:31.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.142+0000 7ff097682740 20 client.25873 trim_cache size 0 max 16384 2024-03-23T23:45:31.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 Processor -- start 2024-03-23T23:45:31.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 -- start start 2024-03-23T23:45:31.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a65e0 con 0x55b46dc7c580 2024-03-23T23:45:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a7f60 con 0x55b46dc7a9f0 2024-03-23T23:45:31.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.160+0000 7ff097682740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a6c40 con 0x55b46dc7bbc0 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff08de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40692/0 (socket says 172.21.15.99:40692) 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 -- 172.21.15.99:0/3656159685 learned_addr learned my addr 172.21.15.99:0/3656159685 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff08e800640 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 -- 172.21.15.99:0/3656159685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 msgr2=0x55b46dc7ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 -- 172.21.15.99:0/3656159685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 msgr2=0x55b46dc7f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc7f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 -- 172.21.15.99:0/3656159685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46dc76850 con 0x55b46dc7bbc0 2024-03-23T23:45:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff087e00640 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff07809b450 tx=0x7ff078086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aabd71fc4c1fcac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff07e600640 1 -- 172.21.15.99:0/3656159685 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07809be50 con 0x55b46dc7bbc0 2024-03-23T23:45:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.161+0000 7ff07e600640 1 -- 172.21.15.99:0/3656159685 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0780b9070 con 0x55b46dc7bbc0 2024-03-23T23:45:31.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff07e600640 1 -- 172.21.15.99:0/3656159685 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780860b0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 -- 172.21.15.99:0/3656159685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 msgr2=0x55b46dc8c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff07809b450 tx=0x7ff078086040 comp rx=0 tx=0).stop 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 -- 172.21.15.99:0/3656159685 shutdown_connections 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc7f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 --2- 172.21.15.99:0/3656159685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc7ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 -- 172.21.15.99:0/3656159685 >> 172.21.15.99:0/3656159685 conn(0x55b46dc75de0 msgr2=0x55b46dc798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 -- 172.21.15.99:0/3656159685 shutdown_connections 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.162+0000 7ff097682740 1 -- 172.21.15.99:0/3656159685 wait complete. 2024-03-23T23:45:31.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.163+0000 7ff097682740 1 -- start start 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.163+0000 7ff097682740 1 Processor -- start 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc8bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a7900 con 0x55b46dc7c580 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a72a0 con 0x55b46dc7a9f0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901a47a0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff097682740 10 client.0 _ll_register_callbacks cb 0x7ff078005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40708/0 (socket says 172.21.15.99:40708) 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 -- 172.21.15.99:0/3728189713 learned_addr learned my addr 172.21.15.99:0/3728189713 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff08de00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc8bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff08e800640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 -- 172.21.15.99:0/3728189713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 msgr2=0x55b46dc8bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc8bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 -- 172.21.15.99:0/3728189713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 msgr2=0x55b46dc88940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc88940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.164+0000 7ff087e00640 1 -- 172.21.15.99:0/3728189713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b46dc89140 con 0x55b46dc7bbc0 2024-03-23T23:45:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff087e00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0780b6040 tx=0x7ff07809f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1919e60c0234fede server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff073400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07809b750 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff078097dc0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0780b9070 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0780a3ef0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff097682740 10 client.25879 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b46dcf5670 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.165+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0780b9070 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff078084ce0 con 0x55b46dc7bbc0 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 10 client.25879 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 10 client.25879 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 20 client.25879 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dcf5f30 0x55b46dcf8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 4 client.25879 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b46dbe6b50 con 0x55b46dcf5f30 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff08de00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dcf5f30 0x55b46dcf8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff08de00640 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dcf5f30 0x55b46dcf8350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59969fbcf9494620 server_cookie=4478bdad13f7a7d7 in_seq=0 out_seq=0 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.166+0000 7ff073400640 10 client.25879 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:45:31.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff073400640 1 -- 172.21.15.99:0/3728189713 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b46dbe6b50 con 0x55b46dcf5f30 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff073400640 10 client.25879 handle_command_reply: tid=0 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 client.25879 shutdown 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 msgr2=0x55b46dc8c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff0780b6040 tx=0x7ff07809f9e0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 shutdown_connections 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55b46dcf5f30 0x55b46dcf8350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b46dc7c580 0x55b46dc88940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b46dc7bbc0 0x55b46dc8c2c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 --2- 172.21.15.99:0/3728189713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b46dc7a9f0 0x55b46dc8bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff073400640 0 client.25879 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.167+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 >> 172.21.15.99:0/3728189713 conn(0x55b46dc75de0 msgr2=0x55b46dc748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 shutdown_connections 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff097682740 1 -- 172.21.15.99:0/3728189713 wait complete. 2024-03-23T23:45:31.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff097682740 20 client.25879 trim_cache size 0 max 16384 2024-03-23T23:45:31.172 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 msgr2=0x7ff0740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 0x7ff0740a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff080006010 tx=0x7ff08002df90 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 msgr2=0x7ff09019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.169+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9f0 tx=0x7ff07800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 shutdown_connections 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff0740a2800 0x7ff0740a4cc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff090105350 0x7ff0901a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff090104990 0x7ff09019a010 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 --2- 172.21.15.99:0/3043645530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0901037a0 0x7ff090199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 >> 172.21.15.99:0/3043645530 conn(0x7ff0900fed10 msgr2=0x7ff090107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 shutdown_connections 2024-03-23T23:45:31.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.170+0000 7ff08f200640 1 -- 172.21.15.99:0/3043645530 wait complete. 2024-03-23T23:45:31.191 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:31 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3893810665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:45:31.194 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:45:31.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T23:45:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:31 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3893810665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.399+0000 7f6856200640 1 -- 172.21.15.99:0/4188584191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850107bc0 msgr2=0x7f6850107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.399+0000 7f6856200640 1 --2- 172.21.15.99:0/4188584191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850107bc0 0x7f6850107fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6844009900 tx=0x7f684402bc50 comp rx=0 tx=0).stop 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 -- 172.21.15.99:0/4188584191 shutdown_connections 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 --2- 172.21.15.99:0/4188584191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850109770 0x7f6850110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 --2- 172.21.15.99:0/4188584191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850108db0 0x7f6850109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 --2- 172.21.15.99:0/4188584191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850107bc0 0x7f6850107fe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 -- 172.21.15.99:0/4188584191 >> 172.21.15.99:0/4188584191 conn(0x7f6850076b70 msgr2=0x7f6850078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 -- 172.21.15.99:0/4188584191 shutdown_connections 2024-03-23T23:45:31.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.400+0000 7f6856200640 1 -- 172.21.15.99:0/4188584191 wait complete. 2024-03-23T23:45:31.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 Processor -- start 2024-03-23T23:45:31.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 -- start start 2024-03-23T23:45:31.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850108db0 0x7f685019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850109770 0x7f68501a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850113880 con 0x7f6850108db0 2024-03-23T23:45:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850113700 con 0x7f6850109770 2024-03-23T23:45:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.401+0000 7f6856200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850113a00 con 0x7f6850107bc0 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850109770 0x7f68501a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40728/0 (socket says 172.21.15.99:40728) 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 -- 172.21.15.99:0/3962575167 learned_addr learned my addr 172.21.15.99:0/3962575167 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f684fe00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850108db0 0x7f685019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 -- 172.21.15.99:0/3962575167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850109770 msgr2=0x7f68501a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850109770 0x7f68501a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 -- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850108db0 msgr2=0x7f685019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850108db0 0x7f685019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 -- 172.21.15.99:0/3962575167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68501a5af0 con 0x7f6850107bc0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.402+0000 7f6854e00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6844005ab0 tx=0x7f684402c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.403+0000 7f684d600640 1 -- 172.21.15.99:0/3962575167 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844041070 con 0x7f6850107bc0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.403+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68501a5d80 con 0x7f6850107bc0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.403+0000 7f684d600640 1 -- 172.21.15.99:0/3962575167 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6844004450 con 0x7f6850107bc0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.403+0000 7f684d600640 1 -- 172.21.15.99:0/3962575167 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844034440 con 0x7f6850107bc0 2024-03-23T23:45:31.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.403+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68501a61f0 con 0x7f6850107bc0 2024-03-23T23:45:31.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.407+0000 7f685a5a9740 1 Processor -- start 2024-03-23T23:45:31.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.407+0000 7f685a5a9740 1 -- start start 2024-03-23T23:45:31.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f684d600640 1 -- 172.21.15.99:0/3962575167 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6844044070 con 0x7f6850107bc0 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddbd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 0x55844ddc3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844dd202c0 con 0x55844ddbcbc0 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844dcc8c90 con 0x55844ddbd580 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.408+0000 7f685a5a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844dd1c360 con 0x55844ddbb9f0 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40734/0 (socket says 172.21.15.99:40734) 2024-03-23T23:45:31.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 -- 172.21.15.99:0/643615859 learned_addr learned my addr 172.21.15.99:0/643615859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f684fe00640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 0x55844ddc3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6854e00640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddbd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 -- 172.21.15.99:0/643615859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 msgr2=0x55844ddc3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 0x55844ddc3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 -- 172.21.15.99:0/643615859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 msgr2=0x55844ddbd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddbd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f6855800640 1 -- 172.21.15.99:0/643615859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55844dced080 con 0x55844ddbb9f0 2024-03-23T23:45:31.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.409+0000 7f684fe00640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 0x55844ddc3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f6855800640 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6848010370 tx=0x7f6848010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6165d9eb46286fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f684d600640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f6842000640 1 -- 172.21.15.99:0/643615859 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848016070 con 0x55844ddbb9f0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f6842000640 1 -- 172.21.15.99:0/643615859 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6848004d20 con 0x55844ddbb9f0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f6854e00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f6842000640 1 -- 172.21.15.99:0/643615859 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848005420 con 0x55844ddbb9f0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.410+0000 7f684d600640 1 -- 172.21.15.99:0/3962575167 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f68440f0690 con 0x7f6850107bc0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 -- 172.21.15.99:0/643615859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 msgr2=0x55844ddbbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6848010370 tx=0x7f6848010830 comp rx=0 tx=0).stop 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f6854e00640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 0x7f68380a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68440b99e0 tx=0x7f68440dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 -- 172.21.15.99:0/643615859 shutdown_connections 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbd580 0x55844ddc3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddbd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 --2- 172.21.15.99:0/643615859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddbbe10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 -- 172.21.15.99:0/643615859 >> 172.21.15.99:0/643615859 conn(0x55844ddb6de0 msgr2=0x55844ddb71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 -- 172.21.15.99:0/643615859 shutdown_connections 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.411+0000 7f685a5a9740 1 -- 172.21.15.99:0/643615859 wait complete. 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.412+0000 7f685a5a9740 1 -- start start 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.412+0000 7f685a5a9740 1 Processor -- start 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddcd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844ddc4590 con 0x55844ddbcbc0 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844dcdc0d0 con 0x55844ddbb9f0 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55844ddc4710 con 0x55844ddbd580 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f685a5a9740 10 client.0 _ll_register_callbacks cb 0x55844dce0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f684fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37982/0 (socket says 172.21.15.99:37982) 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 -- 172.21.15.99:0/4243204658 learned_addr learned my addr 172.21.15.99:0/4243204658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddcd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 -- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 msgr2=0x55844ddcd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddcd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 -- 172.21.15.99:0/4243204658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 msgr2=0x55844ddc6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.413+0000 7f6855800640 1 -- 172.21.15.99:0/4243204658 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55844da343f0 con 0x55844ddbb9f0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f6855800640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6848004ce0 tx=0x7f6848009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd64cea5f519df server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f6836a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848016040 con 0x55844ddbb9f0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6848004d20 con 0x55844ddbb9f0 2024-03-23T23:45:31.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6848005880 con 0x55844ddbb9f0 2024-03-23T23:45:31.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6848003970 con 0x55844ddbb9f0 2024-03-23T23:45:31.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f685a5a9740 10 client.25844 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.414+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55844dafe7d0 con 0x55844ddbb9f0 2024-03-23T23:45:31.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6848004d20 con 0x55844ddbb9f0 2024-03-23T23:45:31.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f6848028040 con 0x55844ddbb9f0 2024-03-23T23:45:31.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 10 client.25844 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 10 client.25844 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 20 client.25844 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844ddd1460 0x55844ddd3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 4 client.25844 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55844dd2d800 con 0x55844ddd1460 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.415+0000 7f684fe00640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844ddd1460 0x55844ddd3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.416+0000 7f684fe00640 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844ddd1460 0x55844ddd3880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52aee737e6e1d999 server_cookie=6c849da73b07145 in_seq=0 out_seq=0 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.416+0000 7f6836a00640 10 client.25844 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.416+0000 7f6836a00640 1 -- 172.21.15.99:0/4243204658 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55844dd2d800 con 0x55844ddd1460 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.416+0000 7f6836a00640 10 client.25844 handle_command_reply: tid=0 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.416+0000 7f685a5a9740 1 client.25844 shutdown 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 msgr2=0x55844ddc5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6848004ce0 tx=0x7f6848009bb0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 shutdown_connections 2024-03-23T23:45:31.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844ddd1460 0x55844ddd3880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddcd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 --2- 172.21.15.99:0/4243204658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddc5cd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 >> 172.21.15.99:0/4243204658 conn(0x55844ddb6de0 msgr2=0x55844ddb6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f6836a00640 0 client.25844 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 shutdown_connections 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 1 -- 172.21.15.99:0/4243204658 wait complete. 2024-03-23T23:45:31.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.417+0000 7f685a5a9740 20 client.25844 trim_cache size 0 max 16384 2024-03-23T23:45:31.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:31 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3893810665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 Processor -- start 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 -- start start 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddd8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501aee00 con 0x55844ddbcbc0 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501b0780 con 0x55844ddbb9f0 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f685a5a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501af460 con 0x55844ddbd580 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.432+0000 7f6855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f6855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:37998/0 (socket says 172.21.15.99:37998) 2024-03-23T23:45:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f6855800640 1 -- 172.21.15.99:0/3905857417 learned_addr learned my addr 172.21.15.99:0/3905857417 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f6855800640 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddd8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 -- 172.21.15.99:0/3905857417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 msgr2=0x55844ddd8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddd8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 -- 172.21.15.99:0/3905857417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 msgr2=0x55844ddd3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 -- 172.21.15.99:0/3905857417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55844ddb7850 con 0x55844ddbcbc0 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f6855800640 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.433+0000 7f684fe00640 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc5f50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55844ddd4090 tx=0x7f683c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f02b965dbab3822 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.434+0000 7f6841600640 1 -- 172.21.15.99:0/3905857417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c005020 con 0x55844ddbcbc0 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.434+0000 7f6841600640 1 -- 172.21.15.99:0/3905857417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f683c0217e0 con 0x55844ddbcbc0 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.434+0000 7f6841600640 1 -- 172.21.15.99:0/3905857417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c004ba0 con 0x55844ddbcbc0 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- 172.21.15.99:0/3905857417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 msgr2=0x55844ddc5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc5f50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55844ddd4090 tx=0x7f683c005f90 comp rx=0 tx=0).stop 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- 172.21.15.99:0/3905857417 shutdown_connections 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbd580 0x55844ddd8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbcbc0 0x55844ddc5f50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 --2- 172.21.15.99:0/3905857417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbb9f0 0x55844ddd3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- 172.21.15.99:0/3905857417 >> 172.21.15.99:0/3905857417 conn(0x55844ddb6de0 msgr2=0x55844ddba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- 172.21.15.99:0/3905857417 shutdown_connections 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- 172.21.15.99:0/3905857417 wait complete. 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.435+0000 7f685a5a9740 1 -- start start 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 Processor -- start 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddc8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 0x55844ddcb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501b0120 con 0x55844ddbd580 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501afac0 con 0x55844ddbcbc0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501acfc0 con 0x55844ddbb9f0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f685a5a9740 10 client.0 _ll_register_callbacks cb 0x7f683c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f6855800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddc8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f684fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f6855800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 0x55844ddcb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f684fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38014/0 (socket says 172.21.15.99:38014) 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.436+0000 7f684fe00640 1 -- 172.21.15.99:0/1569612973 learned_addr learned my addr 172.21.15.99:0/1569612973 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 -- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 msgr2=0x55844ddc8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddc8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 -- 172.21.15.99:0/1569612973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 msgr2=0x55844ddcb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 0x55844ddcb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 -- 172.21.15.99:0/1569612973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55844ddcbce0 con 0x55844ddbcbc0 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f6855800640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 0x55844ddcb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f684fe00640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d20 tx=0x7f683c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840e0734d9c045ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.437+0000 7f6836a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c021da0 con 0x55844ddbcbc0 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f683c004920 con 0x55844ddbcbc0 2024-03-23T23:45:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f683c0217e0 con 0x55844ddbcbc0 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f683c03a070 con 0x55844ddbcbc0 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f685a5a9740 10 client.25850 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55844de2ce00 con 0x55844ddbcbc0 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.438+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f683c0217e0 con 0x55844ddbcbc0 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f683c0037b0 con 0x55844ddbcbc0 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 10 client.25850 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 10 client.25850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:45:31.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 20 client.25850 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844de2d550 0x55844de2f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 4 client.25850 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55844dd27b50 con 0x55844de2d550 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.439+0000 7f6855800640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844de2d550 0x55844de2f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f6836a00640 10 client.25850 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f6855800640 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844de2d550 0x55844de2f990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f0f76a0242e3679 server_cookie=c212b1832a40b06d in_seq=0 out_seq=0 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f6836a00640 1 -- 172.21.15.99:0/1569612973 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55844dd27b50 con 0x55844de2d550 2024-03-23T23:45:31.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f6836a00640 10 client.25850 handle_command_reply: tid=0 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 client.25850 shutdown 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 msgr2=0x55844ddc86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d20 tx=0x7f683c009f70 comp rx=0 tx=0).stop 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 shutdown_connections 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x55844de2d550 0x55844de2f990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55844ddbd580 0x55844ddcb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55844ddbcbc0 0x55844ddc86d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f6836a00640 0 client.25850 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:45:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 --2- 172.21.15.99:0/1569612973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55844ddbb9f0 0x55844ddc8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 >> 172.21.15.99:0/1569612973 conn(0x55844ddb6de0 msgr2=0x55844ddba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 shutdown_connections 2024-03-23T23:45:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.440+0000 7f685a5a9740 1 -- 172.21.15.99:0/1569612973 wait complete. 2024-03-23T23:45:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f685a5a9740 20 client.25850 trim_cache size 0 max 16384 2024-03-23T23:45:31.449 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:45:31.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 msgr2=0x7f68380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 0x7f68380a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68440b99e0 tx=0x7f68440dd040 comp rx=0 tx=0).stop 2024-03-23T23:45:31.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 msgr2=0x7f685019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6844005ab0 tx=0x7f684402c6e0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 shutdown_connections 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6850109770 0x7f68501a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6850108db0 0x7f685019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 --2- 172.21.15.99:0/3962575167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6850107bc0 0x7f685019ded0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 >> 172.21.15.99:0/3962575167 conn(0x7f6850076b70 msgr2=0x7f685010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.441+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 shutdown_connections 2024-03-23T23:45:31.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.442+0000 7f6856200640 1 -- 172.21.15.99:0/3962575167 wait complete. 2024-03-23T23:45:31.460 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:45:31.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T23:45:31.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 -- 172.21.15.99:0/3840973103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc0756e0 msgr2=0x7f45dc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 --2- 172.21.15.99:0/3840973103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc0756e0 0x7f45dc073b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009930 tx=0x7f45cc02bc20 comp rx=0 tx=0).stop 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 -- 172.21.15.99:0/3840973103 shutdown_connections 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 --2- 172.21.15.99:0/3840973103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 --2- 172.21.15.99:0/3840973103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc074100 0x7f45dc074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 --2- 172.21.15.99:0/3840973103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc0756e0 0x7f45dc073b30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 -- 172.21.15.99:0/3840973103 >> 172.21.15.99:0/3840973103 conn(0x7f45dc0fcb10 msgr2=0x7f45dc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 -- 172.21.15.99:0/3840973103 shutdown_connections 2024-03-23T23:45:31.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.656+0000 7f45e1200640 1 -- 172.21.15.99:0/3840973103 wait complete. 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.657+0000 7f45e1200640 1 Processor -- start 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.657+0000 7f45e1200640 1 -- start start 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 0x7f45dc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc1a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc113840 con 0x7f45dc10db70 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1136c0 con 0x7f45dc074100 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45e1200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1139c0 con 0x7f45dc0756e0 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45db400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45db400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38036/0 (socket says 172.21.15.99:38036) 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45db400640 1 -- 172.21.15.99:0/1124338488 learned_addr learned my addr 172.21.15.99:0/1124338488 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45d3e00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 0x7f45dc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.658+0000 7f45dbe00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc1a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 -- 172.21.15.99:0/1124338488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 msgr2=0x7f45dc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 0x7f45dc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 -- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 msgr2=0x7f45dc1a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc1a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 -- 172.21.15.99:0/1124338488 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc1a5b10 con 0x7f45dc074100 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45dbe00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc1a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45d3e00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 0x7f45dc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45db400640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000c00 tx=0x7f45cc005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45d8c00640 1 -- 172.21.15.99:0/1124338488 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc043020 con 0x7f45dc074100 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45d8c00640 1 -- 172.21.15.99:0/1124338488 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45cc0054e0 con 0x7f45dc074100 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc1a5da0 con 0x7f45dc074100 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.659+0000 7f45d8c00640 1 -- 172.21.15.99:0/1124338488 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc003d10 con 0x7f45dc074100 2024-03-23T23:45:31.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.660+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a6260 con 0x7f45dc074100 2024-03-23T23:45:31.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.663+0000 7f45d8c00640 1 -- 172.21.15.99:0/1124338488 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f45cc046020 con 0x7f45dc074100 2024-03-23T23:45:31.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.664+0000 7f45e5577740 1 Processor -- start 2024-03-23T23:45:31.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.664+0000 7f45e5577740 1 -- start start 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f9641e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9649e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f95a62c0 con 0x5565f96419f0 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f954ec90 con 0x5565f9643580 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45e5577740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f95a2360 con 0x5565f9642bc0 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45d8c00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 0x7f45c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45d8c00640 1 -- 172.21.15.99:0/1124338488 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f45cc0f0c90 con 0x7f45dc074100 2024-03-23T23:45:31.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9649e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40798/0 (socket says 172.21.15.99:40798) 2024-03-23T23:45:31.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45db400640 1 -- 172.21.15.99:0/1857961120 learned_addr learned my addr 172.21.15.99:0/1857961120 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9649e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38038/0 (socket says 172.21.15.99:38038) 2024-03-23T23:45:31.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.665+0000 7f45dbe00640 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f9641e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45dbe00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 0x7f45c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 -- 172.21.15.99:0/1857961120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 msgr2=0x5565f9649e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9649e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 -- 172.21.15.99:0/1857961120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 msgr2=0x5565f9641e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f9641e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 -- 172.21.15.99:0/1857961120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f9573080 con 0x5565f9642bc0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45db400640 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0c5820 tx=0x7f45cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e911f81b9264eb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45d0c00640 1 -- 172.21.15.99:0/1857961120 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0bcf10 con 0x5565f9642bc0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45d0c00640 1 -- 172.21.15.99:0/1857961120 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45cc004c80 con 0x5565f9642bc0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.666+0000 7f45d0c00640 1 -- 172.21.15.99:0/1857961120 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0bb8b0 con 0x5565f9642bc0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 -- 172.21.15.99:0/1857961120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 msgr2=0x5565f9643040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0c5820 tx=0x7f45cc009260 comp rx=0 tx=0).stop 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45dbe00640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 0x7f45c00a4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5565f9642ac0 tx=0x7f45d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 -- 172.21.15.99:0/1857961120 shutdown_connections 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9649e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9643040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 --2- 172.21.15.99:0/1857961120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f9641e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 -- 172.21.15.99:0/1857961120 >> 172.21.15.99:0/1857961120 conn(0x5565f963cde0 msgr2=0x5565f963d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 -- 172.21.15.99:0/1857961120 shutdown_connections 2024-03-23T23:45:31.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.667+0000 7f45e5577740 1 -- 172.21.15.99:0/1857961120 wait complete. 2024-03-23T23:45:31.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.668+0000 7f45e5577740 1 -- start start 2024-03-23T23:45:31.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.668+0000 7f45e5577740 1 Processor -- start 2024-03-23T23:45:31.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9642bc0 0x5565f964bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f9652ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f964a350 con 0x5565f9643580 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f95620d0 con 0x5565f9642bc0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f964a4d0 con 0x5565f96419f0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45e5577740 10 client.0 _ll_register_callbacks cb 0x5565f9566e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40814/0 (socket says 172.21.15.99:40814) 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45db400640 1 -- 172.21.15.99:0/889503916 learned_addr learned my addr 172.21.15.99:0/889503916 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45d3e00640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9642bc0 0x5565f964bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.669+0000 7f45dbe00640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f9652ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 -- 172.21.15.99:0/889503916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9642bc0 msgr2=0x5565f964bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9642bc0 0x5565f964bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 -- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 msgr2=0x5565f9652ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f9652ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 -- 172.21.15.99:0/889503916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565f92ba3f0 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45dbe00640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f9652ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45db400640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b9f30 tx=0x7f45cc005690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d14bbc5c4155b30 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45c9600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0c1700 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f45cc0ee820 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45cc004c80 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45cc0a2070 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45e5577740 10 client.25909 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.670+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565f93847d0 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45cc004c80 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f45cc0c58c0 con 0x5565f96419f0 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 10 client.25909 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 10 client.25909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:45:31.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 20 client.25909 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f9657210 0x5565f9659630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 4 client.25909 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:31.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565f95b3800 con 0x5565f9657210 2024-03-23T23:45:31.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.671+0000 7f45d3e00640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f9657210 0x5565f9659630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.672+0000 7f45d3e00640 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f9657210 0x5565f9659630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2860bb5c2bd6e94f server_cookie=f7216a44890d94ee in_seq=0 out_seq=0 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.672+0000 7f45c9600640 10 client.25909 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45c9600640 1 -- 172.21.15.99:0/889503916 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565f95b3800 con 0x5565f9657210 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45c9600640 10 client.25909 handle_command_reply: tid=0 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 client.25909 shutdown 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 msgr2=0x5565f964b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b9f30 tx=0x7f45cc005690 comp rx=0 tx=0).stop 2024-03-23T23:45:31.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 shutdown_connections 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f9657210 0x5565f9659630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f9652ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9642bc0 0x5565f964bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45c9600640 0 client.25909 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 --2- 172.21.15.99:0/889503916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f96419f0 0x5565f964b960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 >> 172.21.15.99:0/889503916 conn(0x5565f963cde0 msgr2=0x5565f963c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 shutdown_connections 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 1 -- 172.21.15.99:0/889503916 wait complete. 2024-03-23T23:45:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.673+0000 7f45e5577740 20 client.25909 trim_cache size 0 max 16384 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 Processor -- start 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 -- start start 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f96419f0 0x5565f965deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f965e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1aedd0 con 0x5565f9643580 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1b0750 con 0x5565f96419f0 2024-03-23T23:45:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45e5577740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1af430 con 0x5565f9642bc0 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40826/0 (socket says 172.21.15.99:40826) 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45dbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f965e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45d3e00640 1 -- 172.21.15.99:0/1141845259 learned_addr learned my addr 172.21.15.99:0/1141845259 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.689+0000 7f45db400640 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f96419f0 0x5565f965deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 -- 172.21.15.99:0/1141845259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f96419f0 msgr2=0x5565f965deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f96419f0 0x5565f965deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 -- 172.21.15.99:0/1141845259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 msgr2=0x5565f965e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f965e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 -- 172.21.15.99:0/1141845259 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f963d850 con 0x5565f9642bc0 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45d3e00640 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5565f963dd50 tx=0x7f45c4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4424e52e93f25770 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45cbe00640 1 -- 172.21.15.99:0/1141845259 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4024070 con 0x5565f9642bc0 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45cbe00640 1 -- 172.21.15.99:0/1141845259 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45c40202f0 con 0x5565f9642bc0 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.690+0000 7f45cbe00640 1 -- 172.21.15.99:0/1141845259 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40209d0 con 0x5565f9642bc0 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 -- 172.21.15.99:0/1141845259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 msgr2=0x5565f9652ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5565f963dd50 tx=0x7f45c4005f90 comp rx=0 tx=0).stop 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 -- 172.21.15.99:0/1141845259 shutdown_connections 2024-03-23T23:45:31.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f9643580 0x5565f965e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f9652ea0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 --2- 172.21.15.99:0/1141845259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f96419f0 0x5565f965deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 -- 172.21.15.99:0/1141845259 >> 172.21.15.99:0/1141845259 conn(0x5565f963cde0 msgr2=0x5565f96408d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 -- 172.21.15.99:0/1141845259 shutdown_connections 2024-03-23T23:45:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.691+0000 7f45e5577740 1 -- 172.21.15.99:0/1141845259 wait complete. 2024-03-23T23:45:31.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 -- start start 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 Processor -- start 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f964f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9652560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1b00f0 con 0x5565f96419f0 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1afa90 con 0x5565f9643580 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1acf90 con 0x5565f9642bc0 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.692+0000 7f45e5577740 10 client.0 _ll_register_callbacks cb 0x7f45c4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45db400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f964f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40840/0 (socket says 172.21.15.99:40840) 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 -- 172.21.15.99:0/526330318 learned_addr learned my addr 172.21.15.99:0/526330318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45dbe00640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9652560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 -- 172.21.15.99:0/526330318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 msgr2=0x5565f9652560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9652560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 -- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 msgr2=0x5565f964f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f964f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 -- 172.21.15.99:0/526330318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565f9652b50 con 0x5565f9642bc0 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45db400640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f964f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45dbe00640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9652560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45d3e00640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45c4004ee0 tx=0x7f45c4004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40d13359c9ea36e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.693+0000 7f45c9600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4024040 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f45c4004030 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565f95adb50 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c4004230 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 10 client.25915 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565f96b34e0 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565f95adb50 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f45c4031030 con 0x5565f9642bc0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 10 client.25915 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 10 client.25915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 20 client.25915 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f96b3db0 0x5565f96b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 4 client.25915 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.694+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5565f952b500 con 0x5565f96b3db0 2024-03-23T23:45:31.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.695+0000 7f45db400640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f96b3db0 0x5565f96b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.695+0000 7f45db400640 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f96b3db0 0x5565f96b61d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c49a826be57a37a server_cookie=645e7feddec93259 in_seq=0 out_seq=0 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.695+0000 7f45c9600640 10 client.25915 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45c9600640 1 -- 172.21.15.99:0/526330318 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5565f952b500 con 0x5565f96b3db0 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45c9600640 10 client.25915 handle_command_reply: tid=0 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 client.25915 shutdown 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 msgr2=0x5565f964f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45c4004ee0 tx=0x7f45c4004f10 comp rx=0 tx=0).stop 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 shutdown_connections 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5565f96b3db0 0x5565f96b61d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565f9643580 0x5565f9652560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565f9642bc0 0x5565f964f5b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 --2- 172.21.15.99:0/526330318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565f96419f0 0x5565f964f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 >> 172.21.15.99:0/526330318 conn(0x5565f963cde0 msgr2=0x5565f9645df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45c9600640 0 client.25915 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:45:31.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 shutdown_connections 2024-03-23T23:45:31.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 1 -- 172.21.15.99:0/526330318 wait complete. 2024-03-23T23:45:31.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.696+0000 7f45e5577740 20 client.25915 trim_cache size 0 max 16384 2024-03-23T23:45:31.702 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:45:31.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 msgr2=0x7f45c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 0x7f45c00a4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5565f9642ac0 tx=0x7f45d4004a40 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 msgr2=0x7f45dc19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000c00 tx=0x7f45cc005d50 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 shutdown_connections 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f45c00a2880 0x7f45c00a4d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45dc10db70 0x7f45dc1a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0756e0 0x7f45dc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 --2- 172.21.15.99:0/1124338488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45dc074100 0x7f45dc19def0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 >> 172.21.15.99:0/1124338488 conn(0x7f45dc0fcb10 msgr2=0x7f45dc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 shutdown_connections 2024-03-23T23:45:31.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.697+0000 7f45e1200640 1 -- 172.21.15.99:0/1124338488 wait complete. 2024-03-23T23:45:31.716 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:45:31.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 -- 172.21.15.99:0/348228305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 msgr2=0x7f5468073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 --2- 172.21.15.99:0/348228305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468073ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f545c009900 tx=0x7f545c02bcd0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 -- 172.21.15.99:0/348228305 shutdown_connections 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 --2- 172.21.15.99:0/348228305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f546810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 --2- 172.21.15.99:0/348228305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 --2- 172.21.15.99:0/348228305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468073ad0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 -- 172.21.15.99:0/348228305 >> 172.21.15.99:0/348228305 conn(0x7f54680fcb10 msgr2=0x7f54680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 -- 172.21.15.99:0/348228305 shutdown_connections 2024-03-23T23:45:31.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.922+0000 7f546e400640 1 -- 172.21.15.99:0/348228305 wait complete. 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.923+0000 7f546e400640 1 Processor -- start 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 -- start start 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f54681a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546810f460 con 0x7f5468109430 2024-03-23T23:45:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546810f2e0 con 0x7f54680740a0 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546e400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546810f5e0 con 0x7f5468075680 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f5467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38086/0 (socket says 172.21.15.99:38086) 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f546d000640 1 -- 172.21.15.99:0/2187170842 learned_addr learned my addr 172.21.15.99:0/2187170842 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.924+0000 7f5467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40850/0 (socket says 172.21.15.99:40850) 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546da00640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f54681a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 -- 172.21.15.99:0/2187170842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 msgr2=0x7f5468199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 -- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 msgr2=0x7f54681a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f54681a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 -- 172.21.15.99:0/2187170842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54681a16c0 con 0x7f54680740a0 2024-03-23T23:45:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546da00640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f54681a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.925+0000 7f546d000640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f545c000c00 tx=0x7f545c02c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.926+0000 7f5465600640 1 -- 172.21.15.99:0/2187170842 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c043020 con 0x7f54680740a0 2024-03-23T23:45:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.926+0000 7f5465600640 1 -- 172.21.15.99:0/2187170842 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545c0054e0 con 0x7f54680740a0 2024-03-23T23:45:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.926+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54681a1950 con 0x7f54680740a0 2024-03-23T23:45:31.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.926+0000 7f5465600640 1 -- 172.21.15.99:0/2187170842 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c003c20 con 0x7f54680740a0 2024-03-23T23:45:31.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.926+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681a1ef0 con 0x7f54680740a0 2024-03-23T23:45:31.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.929+0000 7f5465600640 1 -- 172.21.15.99:0/2187170842 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f545c046020 con 0x7f54680740a0 2024-03-23T23:45:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.930+0000 7f5472629740 1 Processor -- start 2024-03-23T23:45:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.930+0000 7f5472629740 1 -- start start 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560016532c0 con 0x5560016f0580 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560015fbc90 con 0x5560016ee9f0 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5472629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600164f360 con 0x5560016efbc0 2024-03-23T23:45:31.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5465600640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 0x7f54500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40860/0 (socket says 172.21.15.99:40860) 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 -- 172.21.15.99:0/1005645666 learned_addr learned my addr 172.21.15.99:0/1005645666 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5465600640 1 -- 172.21.15.99:0/2187170842 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f545c0f0b60 con 0x7f54680740a0 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546da00640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f5467e00640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546da00640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 0x7f54500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 -- 172.21.15.99:0/1005645666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 msgr2=0x5560016eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 -- 172.21.15.99:0/1005645666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 msgr2=0x5560016f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.931+0000 7f546d000640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f546d000640 1 -- 172.21.15.99:0/1005645666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556001620080 con 0x5560016efbc0 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f5467e00640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f546d000640 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f545c02c210 tx=0x7f545c0901b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170b56107651ab8f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f546da00640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 0x7f54500a4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5560016efac0 tx=0x7f5460009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f545a000640 1 -- 172.21.15.99:0/1005645666 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c004c80 con 0x5560016efbc0 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f545a000640 1 -- 172.21.15.99:0/1005645666 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545c005690 con 0x5560016efbc0 2024-03-23T23:45:31.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.932+0000 7f545a000640 1 -- 172.21.15.99:0/1005645666 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0c5c20 con 0x5560016efbc0 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 -- 172.21.15.99:0/1005645666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 msgr2=0x5560016f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f545c02c210 tx=0x7f545c0901b0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 -- 172.21.15.99:0/1005645666 shutdown_connections 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016f0040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 --2- 172.21.15.99:0/1005645666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 -- 172.21.15.99:0/1005645666 >> 172.21.15.99:0/1005645666 conn(0x5560016e9de0 msgr2=0x5560016ea1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 -- 172.21.15.99:0/1005645666 shutdown_connections 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.933+0000 7f5472629740 1 -- 172.21.15.99:0/1005645666 wait complete. 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.934+0000 7f5472629740 1 -- start start 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.934+0000 7f5472629740 1 Processor -- start 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 0x5560016f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016f0580 0x5560016ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560016f7350 con 0x5560016efbc0 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600160f0d0 con 0x5560016f0580 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560016f74d0 con 0x5560016ee9f0 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5472629740 10 client.0 _ll_register_callbacks cb 0x556001613e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40874/0 (socket says 172.21.15.99:40874) 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 -- 172.21.15.99:0/1785846789 learned_addr learned my addr 172.21.15.99:0/1785846789 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546da00640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016f0580 0x5560016ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f5467e00640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 0x5560016f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 -- 172.21.15.99:0/1785846789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016f0580 msgr2=0x5560016ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016f0580 0x5560016ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 -- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 msgr2=0x5560016f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.935+0000 7f546d000640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 0x5560016f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f546d000640 1 -- 172.21.15.99:0/1785846789 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560013673f0 con 0x5560016ee9f0 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f5467e00640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 0x5560016f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f546d000640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f545c0bd180 tx=0x7f545c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d976e046b1370fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f544ea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0c15d0 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f545c0d61b0 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556001660800 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f545c0c5c20 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f5472629740 10 client.25927 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.936+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5560014317d0 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556001660800 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f545c0c10d0 con 0x5560016ee9f0 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 10 client.25927 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 10 client.25927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 20 client.25927 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001704040 0x5560017064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 4 client.25927 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:31.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55600165ab50 con 0x556001704040 2024-03-23T23:45:31.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.937+0000 7f5467e00640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001704040 0x5560017064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.938+0000 7f5467e00640 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001704040 0x5560017064c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bf9dcb8122dac30 server_cookie=5c522b863c07cdad in_seq=0 out_seq=0 2024-03-23T23:45:31.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.938+0000 7f544ea00640 10 client.25927 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f544ea00640 1 -- 172.21.15.99:0/1785846789 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55600165ab50 con 0x556001704040 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f544ea00640 10 client.25927 handle_command_reply: tid=0 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 client.25927 shutdown 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 msgr2=0x5560016f8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f545c0bd180 tx=0x7f545c005e50 comp rx=0 tx=0).stop 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 shutdown_connections 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001704040 0x5560017064c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016f0580 0x5560016ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016efbc0 0x5560016f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 --2- 172.21.15.99:0/1785846789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016ee9f0 0x5560016f8960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f544ea00640 0 client.25927 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:45:31.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.939+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 >> 172.21.15.99:0/1785846789 conn(0x5560016e9de0 msgr2=0x5560016e9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.940+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 shutdown_connections 2024-03-23T23:45:31.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.940+0000 7f5472629740 1 -- 172.21.15.99:0/1785846789 wait complete. 2024-03-23T23:45:31.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.940+0000 7f5472629740 20 client.25927 trim_cache size 0 max 16384 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.953+0000 7f5472629740 1 Processor -- start 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.953+0000 7f5472629740 1 -- start start 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.953+0000 7f5472629740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.953+0000 7f5472629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.953+0000 7f5472629740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f5472629740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681aaa40 con 0x5560016f0580 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f5472629740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681ac3c0 con 0x5560016ee9f0 2024-03-23T23:45:31.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f5472629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681ab0a0 con 0x5560016efbc0 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38114/0 (socket says 172.21.15.99:38114) 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 -- 172.21.15.99:0/1749847009 learned_addr learned my addr 172.21.15.99:0/1749847009 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f5467e00640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546da00640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 -- 172.21.15.99:0/1749847009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 msgr2=0x5560016ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 -- 172.21.15.99:0/1749847009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 msgr2=0x5560016f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546d000640 1 -- 172.21.15.99:0/1749847009 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560016f6710 con 0x5560016ee9f0 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f546da00640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.954+0000 7f5467e00640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.955+0000 7f546d000640 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f545c0b98e0 tx=0x7f545c02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d23c5da5a97b8db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.955+0000 7f5459600640 1 -- 172.21.15.99:0/1749847009 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0c15d0 con 0x5560016ee9f0 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.955+0000 7f5459600640 1 -- 172.21.15.99:0/1749847009 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545c004c80 con 0x5560016ee9f0 2024-03-23T23:45:31.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.955+0000 7f5459600640 1 -- 172.21.15.99:0/1749847009 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0c10d0 con 0x5560016ee9f0 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- 172.21.15.99:0/1749847009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 msgr2=0x5560016f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f545c0b98e0 tx=0x7f545c02bcd0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- 172.21.15.99:0/1749847009 shutdown_connections 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 --2- 172.21.15.99:0/1749847009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016f2bd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- 172.21.15.99:0/1749847009 >> 172.21.15.99:0/1749847009 conn(0x5560016e9de0 msgr2=0x5560016ed8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- 172.21.15.99:0/1749847009 shutdown_connections 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- 172.21.15.99:0/1749847009 wait complete. 2024-03-23T23:45:31.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.956+0000 7f5472629740 1 -- start start 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 Processor -- start 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681abd60 con 0x5560016f0580 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681ab700 con 0x5560016ee9f0 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681a8c00 con 0x5560016efbc0 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5472629740 10 client.0 _ll_register_callbacks cb 0x7f545c0bd060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f5467e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f546d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38128/0 (socket says 172.21.15.99:38128) 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.957+0000 7f546d000640 1 -- 172.21.15.99:0/1633471082 learned_addr learned my addr 172.21.15.99:0/1633471082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f546da00640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 -- 172.21.15.99:0/1633471082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 msgr2=0x5560016fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 -- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 msgr2=0x5560016feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 -- 172.21.15.99:0/1633471082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560016ff4a0 con 0x5560016efbc0 2024-03-23T23:45:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f546d000640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5467e00640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016fbe90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556001706cd0 tx=0x7f5454004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5980e214d541cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f544ea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54540035f0 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5454003790 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.958+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f545c004c80 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5454003990 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 10 client.25933 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55600175fe80 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f545c004c80 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5454025cd0 con 0x5560016efbc0 2024-03-23T23:45:31.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 10 client.25933 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 10 client.25933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 20 client.25933 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001760770 0x556001762bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 4 client.25933 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.959+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5560015d8500 con 0x556001760770 2024-03-23T23:45:31.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.960+0000 7f546d000640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001760770 0x556001762bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.960+0000 7f546d000640 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001760770 0x556001762bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2184722876b41d3d server_cookie=a728b1bca4b98ea4 in_seq=0 out_seq=0 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.960+0000 7f544ea00640 10 client.25933 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f544ea00640 1 -- 172.21.15.99:0/1633471082 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5560015d8500 con 0x556001760770 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f544ea00640 10 client.25933 handle_command_reply: tid=0 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 client.25933 shutdown 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 msgr2=0x5560016fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016fbe90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556001706cd0 tx=0x7f5454004560 comp rx=0 tx=0).stop 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 shutdown_connections 2024-03-23T23:45:31.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x556001760770 0x556001762bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560016f0580 0x5560016feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560016efbc0 0x5560016fbe90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f544ea00640 0 client.25933 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 --2- 172.21.15.99:0/1633471082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560016ee9f0 0x5560016fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 >> 172.21.15.99:0/1633471082 conn(0x5560016e9de0 msgr2=0x5560016e88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.961+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 shutdown_connections 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f5472629740 1 -- 172.21.15.99:0/1633471082 wait complete. 2024-03-23T23:45:31.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f5472629740 20 client.25933 trim_cache size 0 max 16384 2024-03-23T23:45:31.968 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:45:31.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 msgr2=0x7f54500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 0x7f54500a4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5560016efac0 tx=0x7f5460009500 comp rx=0 tx=0).stop 2024-03-23T23:45:31.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 msgr2=0x7f5468199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.962+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f545c000c00 tx=0x7f545c02c6f0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 shutdown_connections 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f54500a2880 0x7f54500a4d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5468109430 0x7f54681a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468075680 0x7f5468199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 --2- 172.21.15.99:0/2187170842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54680740a0 0x7f5468199aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 >> 172.21.15.99:0/2187170842 conn(0x7f54680fcb10 msgr2=0x7f54681053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 shutdown_connections 2024-03-23T23:45:31.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:31.963+0000 7f546e400640 1 -- 172.21.15.99:0/2187170842 wait complete. 2024-03-23T23:45:31.981 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:45:31.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T23:45:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.208+0000 7f8ba6200640 1 -- 172.21.15.99:0/535097901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba01038e0 msgr2=0x7f8ba0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.208+0000 7f8ba6200640 1 --2- 172.21.15.99:0/535097901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba01038e0 0x7f8ba0103d00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009860 tx=0x7f8b9002bb70 comp rx=0 tx=0).stop 2024-03-23T23:45:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 -- 172.21.15.99:0/535097901 shutdown_connections 2024-03-23T23:45:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 --2- 172.21.15.99:0/535097901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba0105490 0x7f8ba010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 --2- 172.21.15.99:0/535097901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 --2- 172.21.15.99:0/535097901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba01038e0 0x7f8ba0103d00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 -- 172.21.15.99:0/535097901 >> 172.21.15.99:0/535097901 conn(0x7f8ba00fee50 msgr2=0x7f8ba0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 -- 172.21.15.99:0/535097901 shutdown_connections 2024-03-23T23:45:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.209+0000 7f8ba6200640 1 -- 172.21.15.99:0/535097901 wait complete. 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.210+0000 7f8ba6200640 1 Processor -- start 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.210+0000 7f8ba6200640 1 -- start start 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba01038e0 0x7f8ba0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba0105490 0x7f8ba01a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba010f530 con 0x7f8ba01038e0 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba010f3b0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8ba6200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba010f6b0 con 0x7f8ba0105490 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba01038e0 0x7f8ba0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba0105490 0x7f8ba01a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38138/0 (socket says 172.21.15.99:38138) 2024-03-23T23:45:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 -- 172.21.15.99:0/2866757034 learned_addr learned my addr 172.21.15.99:0/2866757034 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 -- 172.21.15.99:0/2866757034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba0105490 msgr2=0x7f8ba01a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba0105490 0x7f8ba01a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 -- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba01038e0 msgr2=0x7f8ba0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.211+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba01038e0 0x7f8ba0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8b9ea00640 1 -- 172.21.15.99:0/2866757034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba01a17f0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9c0 tx=0x7f8b8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8b97e00640 1 -- 172.21.15.99:0/2866757034 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00cde0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8b97e00640 1 -- 172.21.15.99:0/2866757034 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c0035f0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8b97e00640 1 -- 172.21.15.99:0/2866757034 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c010690 con 0x7f8ba0104ad0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba01a1ae0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.212+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba01a1fe0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.216+0000 7f8ba8f76740 1 Processor -- start 2024-03-23T23:45:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.216+0000 7f8ba8f76740 1 -- start start 2024-03-23T23:45:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x562569439c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x562569441c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256939e2c0 con 0x56256943b3a0 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562569346c90 con 0x562569439810 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256939a360 con 0x56256943a9e0 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x562569439c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40914/0 (socket says 172.21.15.99:40914) 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 -- 172.21.15.99:0/3581413930 learned_addr learned my addr 172.21.15.99:0/3581413930 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b97e00640 1 -- 172.21.15.99:0/2866757034 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8b8c0037a0 con 0x7f8ba0104ad0 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 -- 172.21.15.99:0/3581413930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 msgr2=0x562569439c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x562569439c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.217+0000 7f8b9f400640 1 -- 172.21.15.99:0/3581413930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 msgr2=0x562569441c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x562569441c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b9f400640 1 -- 172.21.15.99:0/3581413930 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256936b080 con 0x56256943a9e0 2024-03-23T23:45:32.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x562569439c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:32.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b900057c0 tx=0x7f8b90009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73109cdac3146ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b8be00640 1 -- 172.21.15.99:0/3581413930 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90043020 con 0x56256943a9e0 2024-03-23T23:45:32.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b8be00640 1 -- 172.21.15.99:0/3581413930 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b90034db0 con 0x56256943a9e0 2024-03-23T23:45:32.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.218+0000 7f8b8be00640 1 -- 172.21.15.99:0/3581413930 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90039040 con 0x56256943a9e0 2024-03-23T23:45:32.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.219+0000 7f8b97e00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 0x7f8b840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.219+0000 7f8b97e00640 1 -- 172.21.15.99:0/2866757034 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8b8c0ced30 con 0x7f8ba0104ad0 2024-03-23T23:45:32.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.219+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3581413930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 msgr2=0x56256943ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.219+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b900057c0 tx=0x7f8b90009490 comp rx=0 tx=0).stop 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.219+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 0x7f8b840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3581413930 shutdown_connections 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x562569441c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256943ae60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3581413930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x562569439c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3581413930 >> 172.21.15.99:0/3581413930 conn(0x562569434c00 msgr2=0x562569435000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3581413930 shutdown_connections 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3581413930 wait complete. 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 0x7f8b840a4d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56256943a8e0 tx=0x7f8b98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.220+0000 7f8ba8f76740 1 -- start start 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 Processor -- start 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 0x562569443ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 0x56256944b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562569442350 con 0x562569439810 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256935a0d0 con 0x56256943b3a0 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.221+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625694424d0 con 0x56256943a9e0 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8ba8f76740 10 client.0 _ll_register_callbacks cb 0x56256935ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 0x56256944b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40916/0 (socket says 172.21.15.99:40916) 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9ea00640 1 -- 172.21.15.99:0/386056646 learned_addr learned my addr 172.21.15.99:0/386056646 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:32.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9f400640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 0x562569443ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 -- 172.21.15.99:0/386056646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 msgr2=0x562569443ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 -- 172.21.15.99:0/386056646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 msgr2=0x562569443ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 0x562569443ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9fe00640 1 -- 172.21.15.99:0/386056646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625690b23f0 con 0x56256943b3a0 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9f400640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 0x562569443ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:32.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.222+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 0x56256944b030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005fd0 tx=0x7f8b98004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb03566f4435de58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8b88c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004b20 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8b98004cc0 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b90034db0 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b9800f860 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8ba8f76740 10 client.25898 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.223+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56256917c7d0 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b90034db0 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8b98026070 con 0x56256943b3a0 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8ba8f76740 10 client.25898 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8ba8f76740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8ba8f76740 20 client.25898 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:32.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.224+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x562569451a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.225+0000 7f8ba8f76740 4 client.25898 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.225+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625693ab800 con 0x56256944f5d0 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.225+0000 7f8b9f400640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x562569451a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.225+0000 7f8b9f400640 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x562569451a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512c14e405700fc4 server_cookie=6002524d70cadbfb in_seq=0 out_seq=0 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.225+0000 7f8b88c00640 10 client.25898 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.226+0000 7f8b88c00640 1 -- 172.21.15.99:0/386056646 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5625693ab800 con 0x56256944f5d0 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.226+0000 7f8b88c00640 10 client.25898 handle_command_reply: tid=0 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.226+0000 7f8ba8f76740 1 client.25898 shutdown 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.226+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 msgr2=0x56256944b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.226+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 0x56256944b030 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005fd0 tx=0x7f8b98004ab0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 shutdown_connections 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x562569451a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943b3a0 0x56256944b030 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8b88c00640 0 client.25898 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x562569443ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/386056646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562569439810 0x562569443ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 >> 172.21.15.99:0/386056646 conn(0x562569434c00 msgr2=0x562569433eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 shutdown_connections 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 1 -- 172.21.15.99:0/386056646 wait complete. 2024-03-23T23:45:32.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.227+0000 7f8ba8f76740 20 client.25898 trim_cache size 0 max 16384 2024-03-23T23:45:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 Processor -- start 2024-03-23T23:45:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 -- start start 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943a9e0 0x56256944aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x56256943dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01aac50 con 0x56256943b3a0 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01ac5d0 con 0x56256943a9e0 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01ab2b0 con 0x562569439810 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40928/0 (socket says 172.21.15.99:40928) 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8b9f400640 1 -- 172.21.15.99:0/3453877998 learned_addr learned my addr 172.21.15.99:0/3453877998 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.248+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943a9e0 0x56256944aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x56256943dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 -- 172.21.15.99:0/3453877998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943a9e0 msgr2=0x56256944aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943a9e0 0x56256944aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 -- 172.21.15.99:0/3453877998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 msgr2=0x56256943dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x56256943dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 -- 172.21.15.99:0/3453877998 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562569441600 con 0x562569439810 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x56256943dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b9f400640 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009860 tx=0x7f8b90000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=351af7242645202 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b8b400640 1 -- 172.21.15.99:0/3453877998 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90039040 con 0x562569439810 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b8b400640 1 -- 172.21.15.99:0/3453877998 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b90034db0 con 0x562569439810 2024-03-23T23:45:32.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.249+0000 7f8b8b400640 1 -- 172.21.15.99:0/3453877998 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90043040 con 0x562569439810 2024-03-23T23:45:32.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3453877998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 msgr2=0x5625694430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009860 tx=0x7f8b90000bc0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3453877998 shutdown_connections 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x56256943dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56256943a9e0 0x56256944aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/3453877998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562569439810 0x5625694430d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3453877998 >> 172.21.15.99:0/3453877998 conn(0x562569434c00 msgr2=0x5625694386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.250+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3453877998 shutdown_connections 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.251+0000 7f8ba8f76740 1 -- 172.21.15.99:0/3453877998 wait complete. 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.251+0000 7f8ba8f76740 1 -- start start 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.251+0000 7f8ba8f76740 1 Processor -- start 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x5625694472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01abf70 con 0x56256943b3a0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01ab910 con 0x562569439810 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01a8e10 con 0x56256943a9e0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8ba8f76740 10 client.0 _ll_register_callbacks cb 0x7f8b980114a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38180/0 (socket says 172.21.15.99:38180) 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 -- 172.21.15.99:0/4273311405 learned_addr learned my addr 172.21.15.99:0/4273311405 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40930/0 (socket says 172.21.15.99:40930) 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x5625694472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 -- 172.21.15.99:0/4273311405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 msgr2=0x56256944ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 -- 172.21.15.99:0/4273311405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 msgr2=0x5625694472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x5625694472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9f400640 1 -- 172.21.15.99:0/4273311405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562569447960 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.252+0000 7f8b9fe00640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x5625694472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8b9f400640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b9002c150 tx=0x7f8b9002c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a843c14d4f83010b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8b88c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90057070 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8b90034740 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b90034db0 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b90039050 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8ba8f76740 10 client.25904 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.253+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625694aba80 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b90034db0 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8b900328b0 con 0x562569439810 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 10 client.25904 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:32.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 20 client.25904 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x5625694aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 4 client.25904 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625693a5b50 con 0x56256944f5d0 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.254+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x5625694aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8b9ea00640 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x5625694aded0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfb46210b867d13 server_cookie=c8af53253a3d5609 in_seq=0 out_seq=0 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8b88c00640 10 client.25904 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8b88c00640 1 -- 172.21.15.99:0/4273311405 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625693a5b50 con 0x56256944f5d0 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8b88c00640 10 client.25904 handle_command_reply: tid=0 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 client.25904 shutdown 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 msgr2=0x56256944a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b9002c150 tx=0x7f8b9002c510 comp rx=0 tx=0).stop 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 shutdown_connections 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56256944f5d0 0x5625694aded0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56256943b3a0 0x5625694472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56256943a9e0 0x56256944ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 --2- 172.21.15.99:0/4273311405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562569439810 0x56256944a800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8b88c00640 0 client.25904 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 >> 172.21.15.99:0/4273311405 conn(0x562569434c00 msgr2=0x5625694386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 shutdown_connections 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 1 -- 172.21.15.99:0/4273311405 wait complete. 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.255+0000 7f8ba8f76740 20 client.25904 trim_cache size 0 max 16384 2024-03-23T23:45:32.260 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 msgr2=0x7f8b840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 0x7f8b840a4d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56256943a8e0 tx=0x7f8b98009500 comp rx=0 tx=0).stop 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 msgr2=0x7f8ba019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9c0 tx=0x7f8b8c00ee80 comp rx=0 tx=0).stop 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 shutdown_connections 2024-03-23T23:45:32.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8b840a2880 0x7f8b840a4d40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba0105490 0x7f8ba01a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ba0104ad0 0x7f8ba019a110 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 --2- 172.21.15.99:0/2866757034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ba01038e0 0x7f8ba0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.256+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 >> 172.21.15.99:0/2866757034 conn(0x7f8ba00fee50 msgr2=0x7f8ba0107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.257+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 shutdown_connections 2024-03-23T23:45:32.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:32.257+0000 7f8ba6200640 1 -- 172.21.15.99:0/2866757034 wait complete. 2024-03-23T23:45:32.276 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:32 smithi099 ceph-mon[31715]: pgmap v449: 97 pgs: 97 active+clean; 8.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 685 KiB/s wr, 11 op/s 2024-03-23T23:45:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:32 smithi149 ceph-mon[36083]: pgmap v449: 97 pgs: 97 active+clean; 8.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 685 KiB/s wr, 11 op/s 2024-03-23T23:45:32.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:32 smithi174 ceph-mon[37002]: pgmap v449: 97 pgs: 97 active+clean; 8.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 685 KiB/s wr, 11 op/s 2024-03-23T23:45:33.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T23:45:33.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:45:33.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T23:45:33.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 -- 172.21.15.99:0/1879224064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 msgr2=0x7f8448104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 --2- 172.21.15.99:0/1879224064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f8448104f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8438009900 tx=0x7f843802bc50 comp rx=0 tx=0).stop 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 -- 172.21.15.99:0/1879224064 shutdown_connections 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 --2- 172.21.15.99:0/1879224064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f844810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 --2- 172.21.15.99:0/1879224064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f8448104f30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 --2- 172.21.15.99:0/1879224064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 -- 172.21.15.99:0/1879224064 >> 172.21.15.99:0/1879224064 conn(0x7f84480fee50 msgr2=0x7f8448101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 -- 172.21.15.99:0/1879224064 shutdown_connections 2024-03-23T23:45:33.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.522+0000 7f8450800640 1 -- 172.21.15.99:0/1879224064 wait complete. 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 Processor -- start 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 -- start start 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f84481a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844810f520 con 0x7f84481038c0 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844810f3a0 con 0x7f8448105470 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.523+0000 7f8450800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844810f6a0 con 0x7f8448104ab0 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40948/0 (socket says 172.21.15.99:40948) 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 -- 172.21.15.99:0/3960322983 learned_addr learned my addr 172.21.15.99:0/3960322983 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844e800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f84481a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 -- 172.21.15.99:0/3960322983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 msgr2=0x7f84481a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f84481a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 -- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 msgr2=0x7f8448199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844d400640 1 -- 172.21.15.99:0/3960322983 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84481a18c0 con 0x7f8448104ab0 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844e800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f84481a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.524+0000 7f844de00640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f844d400640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8438009230 tx=0x7f8438005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f8442a00640 1 -- 172.21.15.99:0/3960322983 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438042020 con 0x7f8448104ab0 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f8442a00640 1 -- 172.21.15.99:0/3960322983 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84380044d0 con 0x7f8448104ab0 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f8442a00640 1 -- 172.21.15.99:0/3960322983 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843803e460 con 0x7f8448104ab0 2024-03-23T23:45:33.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84481a1b50 con 0x7f8448104ab0 2024-03-23T23:45:33.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.525+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84481a20a0 con 0x7f8448104ab0 2024-03-23T23:45:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 Processor -- start 2024-03-23T23:45:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 -- start start 2024-03-23T23:45:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 0x55a28af194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af1ec00 0x55a28af21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28ae7eeb0 con 0x55a28af190a0 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28ae27a60 con 0x55a28af19a90 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.529+0000 7f8453609740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28ae7af50 con 0x55a28af1ec00 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38212/0 (socket says 172.21.15.99:38212) 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 -- 172.21.15.99:0/2548891628 learned_addr learned my addr 172.21.15.99:0/2548891628 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844e800640 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 0x55a28af194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f8442a00640 1 -- 172.21.15.99:0/3960322983 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8438045020 con 0x7f8448104ab0 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 -- 172.21.15.99:0/2548891628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af1ec00 msgr2=0x55a28af21090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:45:33.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af1ec00 0x55a28af21090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 -- 172.21.15.99:0/2548891628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 msgr2=0x55a28af194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 0x55a28af194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844de00640 1 -- 172.21.15.99:0/2548891628 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a28ac5b7e0 con 0x55a28af19a90 2024-03-23T23:45:33.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.530+0000 7f844e800640 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 0x55a28af194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.531+0000 7f844de00640 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f843c0114f0 tx=0x7f843c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9e1c8d2453770b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.531+0000 7f8437000640 1 -- 172.21.15.99:0/2548891628 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c004310 con 0x55a28af19a90 2024-03-23T23:45:33.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.531+0000 7f8437000640 1 -- 172.21.15.99:0/2548891628 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f843c0044b0 con 0x55a28af19a90 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.531+0000 7f8437000640 1 -- 172.21.15.99:0/2548891628 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c003850 con 0x55a28af19a90 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8442a00640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 0x7f84300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 -- 172.21.15.99:0/2548891628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 msgr2=0x55a28af1e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f843c0114f0 tx=0x7f843c0119b0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8442a00640 1 -- 172.21.15.99:0/3960322983 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f84380f0b70 con 0x7f8448104ab0 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f844e800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 0x7f84300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 -- 172.21.15.99:0/2548891628 shutdown_connections 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af1ec00 0x55a28af21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af19a90 0x55a28af1e590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 --2- 172.21.15.99:0/2548891628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af190a0 0x55a28af194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 -- 172.21.15.99:0/2548891628 >> 172.21.15.99:0/2548891628 conn(0x55a28af14610 msgr2=0x55a28af14a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 -- 172.21.15.99:0/2548891628 shutdown_connections 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.532+0000 7f8453609740 1 -- 172.21.15.99:0/2548891628 wait complete. 2024-03-23T23:45:33.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.533+0000 7f844e800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 0x7f84300a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55a28af1a2f0 tx=0x7f84440098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.533+0000 7f8453609740 1 -- start start 2024-03-23T23:45:33.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.533+0000 7f8453609740 1 Processor -- start 2024-03-23T23:45:33.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28af21bb0 con 0x55a28af19a90 2024-03-23T23:45:33.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28ae3aeb0 con 0x55a28af1ec00 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28af21d30 con 0x55a28af190a0 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f8453609740 10 client.0 _ll_register_callbacks cb 0x55a28ae2f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f844e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f844d400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f844e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38220/0 (socket says 172.21.15.99:38220) 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.534+0000 7f844e800640 1 -- 172.21.15.99:0/2655767833 learned_addr learned my addr 172.21.15.99:0/2655767833 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 -- 172.21.15.99:0/2655767833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 msgr2=0x55a28af232b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844de00640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 -- 172.21.15.99:0/2655767833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 msgr2=0x55a28af237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 -- 172.21.15.99:0/2655767833 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a28acbbf70 con 0x55a28af1ec00 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844d400640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f844e800640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8444009bd0 tx=0x7f8444004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a258923a4071ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.535+0000 7f842b400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844401d970 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f843c0044b0 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f844401db10 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f844401dd10 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f8453609740 10 client.25916 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a28ab53170 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.536+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f843c0044b0 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f84440345e0 con 0x55a28af1ec00 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 10 client.25916 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 20 client.25916 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af2f0d0 0x55a28af31510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 4 client.25916 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:33.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a28ae8c3f0 con 0x55a28af2f0d0 2024-03-23T23:45:33.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.537+0000 7f844de00640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af2f0d0 0x55a28af31510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.538+0000 7f844de00640 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af2f0d0 0x55a28af31510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe482aaa6eebca server_cookie=b3272e9dd63118a1 in_seq=0 out_seq=0 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.538+0000 7f842b400640 10 client.25916 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.538+0000 7f842b400640 1 -- 172.21.15.99:0/2655767833 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a28ae8c3f0 con 0x55a28af2f0d0 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.538+0000 7f842b400640 10 client.25916 handle_command_reply: tid=0 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 client.25916 shutdown 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 msgr2=0x55a28af2a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8444009bd0 tx=0x7f8444004a40 comp rx=0 tx=0).stop 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 shutdown_connections 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af2f0d0 0x55a28af31510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af2a830 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 --2- 172.21.15.99:0/2655767833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 >> 172.21.15.99:0/2655767833 conn(0x55a28af14610 msgr2=0x55a28af13830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f842b400640 0 client.25916 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 shutdown_connections 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 1 -- 172.21.15.99:0/2655767833 wait complete. 2024-03-23T23:45:33.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.539+0000 7f8453609740 20 client.25916 trim_cache size 0 max 16384 2024-03-23T23:45:33.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 Processor -- start 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 -- start start 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af2a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af1a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481aac60 con 0x55a28af19a90 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481ac5e0 con 0x55a28af1ec00 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.567+0000 7f8453609740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481ab2c0 con 0x55a28af190a0 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40972/0 (socket says 172.21.15.99:40972) 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 -- 172.21.15.99:0/4007510550 learned_addr learned my addr 172.21.15.99:0/4007510550 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844e800640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af1a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844d400640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af2a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 -- 172.21.15.99:0/4007510550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 msgr2=0x55a28af1a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af1a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 -- 172.21.15.99:0/4007510550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 msgr2=0x55a28af2a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af2a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 -- 172.21.15.99:0/4007510550 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a28af0fc20 con 0x55a28af190a0 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844e800640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af1a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.568+0000 7f844de00640 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f843c0114c0 tx=0x7f843c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a02f5f328b6b0d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8436600640 1 -- 172.21.15.99:0/4007510550 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c02a2d0 con 0x55a28af190a0 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8436600640 1 -- 172.21.15.99:0/4007510550 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f843c004260 con 0x55a28af190a0 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8436600640 1 -- 172.21.15.99:0/4007510550 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c00f560 con 0x55a28af190a0 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 -- 172.21.15.99:0/4007510550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 msgr2=0x55a28af17f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f843c0114c0 tx=0x7f843c0096c0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 -- 172.21.15.99:0/4007510550 shutdown_connections 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af1a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af2a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 --2- 172.21.15.99:0/4007510550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af17f80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 -- 172.21.15.99:0/4007510550 >> 172.21.15.99:0/4007510550 conn(0x55a28af14610 msgr2=0x55a28af17870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 -- 172.21.15.99:0/4007510550 shutdown_connections 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.569+0000 7f8453609740 1 -- 172.21.15.99:0/4007510550 wait complete. 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 -- start start 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 Processor -- start 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af28930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481abf80 con 0x55a28af19a90 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481ab920 con 0x55a28af1ec00 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.570+0000 7f8453609740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481a8e20 con 0x55a28af190a0 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f8453609740 10 client.0 _ll_register_callbacks cb 0x7f8444009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40978/0 (socket says 172.21.15.99:40978) 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 -- 172.21.15.99:0/2717599659 learned_addr learned my addr 172.21.15.99:0/2717599659 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844d400640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af28930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844e800640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 -- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 msgr2=0x55a28af298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 -- 172.21.15.99:0/2717599659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 msgr2=0x55a28af28930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af28930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 -- 172.21.15.99:0/2717599659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a28af15080 con 0x55a28af190a0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844d400640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af28930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f844de00640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f843c009de0 tx=0x7f843c0046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d857cbf377cc6f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.571+0000 7f842b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c03a060 con 0x55a28af190a0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f843c030070 con 0x55a28af190a0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f843c004260 con 0x55a28af190a0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f843c02ae60 con 0x55a28af190a0 2024-03-23T23:45:33.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f8453609740 10 client.25963 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a28af8acd0 con 0x55a28af190a0 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f843c004260 con 0x55a28af190a0 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.572+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f843c00fa50 con 0x55a28af190a0 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 10 client.25963 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 20 client.25963 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af8b390 0x55a28af8d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 4 client.25963 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a28af230a0 con 0x55a28af8b390 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f844d400640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af8b390 0x55a28af8d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f844d400640 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af8b390 0x55a28af8d7b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a285c9ff7116534 server_cookie=2284d25472066efc in_seq=0 out_seq=0 2024-03-23T23:45:33.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.573+0000 7f842b400640 10 client.25963 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f842b400640 1 -- 172.21.15.99:0/2717599659 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a28af230a0 con 0x55a28af8b390 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f842b400640 10 client.25963 handle_command_reply: tid=0 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 client.25963 shutdown 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 msgr2=0x55a28af283f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f843c009de0 tx=0x7f843c0046a0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 shutdown_connections 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55a28af8b390 0x55a28af8d7b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f842b400640 0 client.25963 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a28af1ec00 0x55a28af298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a28af19a90 0x55a28af28930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 --2- 172.21.15.99:0/2717599659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a28af190a0 0x55a28af283f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 >> 172.21.15.99:0/2717599659 conn(0x55a28af14610 msgr2=0x55a28af18100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 shutdown_connections 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 1 -- 172.21.15.99:0/2717599659 wait complete. 2024-03-23T23:45:33.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.575+0000 7f8453609740 20 client.25963 trim_cache size 0 max 16384 2024-03-23T23:45:33.581 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:45:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-23T23:45:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "e2da8d31-f594-4253-b3f3-1ac31001ca5d", 2024-03-23T23:45:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-23T23:45:33.581 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 msgr2=0x7f84300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 0x7f84300a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55a28af1a2f0 tx=0x7f84440098a0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 msgr2=0x7f844819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8438009230 tx=0x7f8438005d50 comp rx=0 tx=0).stop 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 shutdown_connections 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f84300a2800 0x7f84300a4cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8448105470 0x7f84481a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8448104ab0 0x7f844819a1c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 --2- 172.21.15.99:0/3960322983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84481038c0 0x7f8448199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.576+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 >> 172.21.15.99:0/3960322983 conn(0x7f84480fee50 msgr2=0x7f8448107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.577+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 shutdown_connections 2024-03-23T23:45:33.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.577+0000 7f8450800640 1 -- 172.21.15.99:0/3960322983 wait complete. 2024-03-23T23:45:33.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:45:33.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 -- 172.21.15.99:0/65145332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 msgr2=0x7ff688106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 --2- 172.21.15.99:0/65145332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688106d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff674009900 tx=0x7ff67402bc40 comp rx=0 tx=0).stop 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 -- 172.21.15.99:0/65145332 shutdown_connections 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 --2- 172.21.15.99:0/65145332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff6881096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 --2- 172.21.15.99:0/65145332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688106d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 --2- 172.21.15.99:0/65145332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 0x7ff6881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.838+0000 7ff68ec00640 1 -- 172.21.15.99:0/65145332 >> 172.21.15.99:0/65145332 conn(0x7ff6880fba10 msgr2=0x7ff6880fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.839+0000 7ff68ec00640 1 -- 172.21.15.99:0/65145332 shutdown_connections 2024-03-23T23:45:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.839+0000 7ff68ec00640 1 -- 172.21.15.99:0/65145332 wait complete. 2024-03-23T23:45:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.840+0000 7ff68ec00640 1 Processor -- start 2024-03-23T23:45:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.840+0000 7ff68ec00640 1 -- start start 2024-03-23T23:45:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.840+0000 7ff68ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 0x7ff6881978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.840+0000 7ff68ec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff68819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68810d260 con 0x7ff688102000 2024-03-23T23:45:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68810d0e0 con 0x7ff688107260 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68810d3e0 con 0x7ff688104930 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:40992/0 (socket says 172.21.15.99:40992) 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 -- 172.21.15.99:0/3907898596 learned_addr learned my addr 172.21.15.99:0/3907898596 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68e200640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff68819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68d800640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 0x7ff6881978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 -- 172.21.15.99:0/3907898596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 msgr2=0x7ff68819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff68819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 -- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 msgr2=0x7ff6881978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 0x7ff6881978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68ce00640 1 -- 172.21.15.99:0/3907898596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68819f4b0 con 0x7ff688104930 2024-03-23T23:45:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.841+0000 7ff68e200640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff68819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff67402efd0 tx=0x7ff67402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff686000640 1 -- 172.21.15.99:0/3907898596 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674041070 con 0x7ff688104930 2024-03-23T23:45:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff686000640 1 -- 172.21.15.99:0/3907898596 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674004450 con 0x7ff688104930 2024-03-23T23:45:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68819f740 con 0x7ff688104930 2024-03-23T23:45:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff686000640 1 -- 172.21.15.99:0/3907898596 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674003e50 con 0x7ff688104930 2024-03-23T23:45:33.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.842+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68819fc50 con 0x7ff688104930 2024-03-23T23:45:33.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.845+0000 7ff692fbb740 1 Processor -- start 2024-03-23T23:45:33.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.845+0000 7ff692fbb740 1 -- start start 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322f5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f32255250 con 0x564f322f0a40 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f321fda30 con 0x564f322f0050 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff692fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f322512f0 con 0x564f322f5bb0 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38270/0 (socket says 172.21.15.99:38270) 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 -- 172.21.15.99:0/167723121 learned_addr learned my addr 172.21.15.99:0/167723121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68d800640 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 -- 172.21.15.99:0/167723121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 msgr2=0x564f322f8040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:45:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322f8040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.846+0000 7ff68e200640 1 -- 172.21.15.99:0/167723121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 msgr2=0x564f322f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff68e200640 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff68e200640 1 -- 172.21.15.99:0/167723121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f321e8690 con 0x564f322f0050 2024-03-23T23:45:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff686000640 1 -- 172.21.15.99:0/3907898596 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff674044070 con 0x7ff688104930 2024-03-23T23:45:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff68e200640 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff680011540 tx=0x7ff680011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485ca362d263f2cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff67aa00640 1 -- 172.21.15.99:0/167723121 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000f5c0 con 0x564f322f0050 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff67aa00640 1 -- 172.21.15.99:0/167723121 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff68000fc40 con 0x564f322f0050 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.847+0000 7ff67aa00640 1 -- 172.21.15.99:0/167723121 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680019d20 con 0x564f322f0050 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff686000640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 0x7ff6700a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 -- 172.21.15.99:0/167723121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 msgr2=0x564f322f0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff680011540 tx=0x7ff680011a00 comp rx=0 tx=0).stop 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 -- 172.21.15.99:0/167723121 shutdown_connections 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff686000640 1 -- 172.21.15.99:0/3907898596 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff6740f05f0 con 0x7ff688104930 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 --2- 172.21.15.99:0/167723121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322f0470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 -- 172.21.15.99:0/167723121 >> 172.21.15.99:0/167723121 conn(0x564f322b8200 msgr2=0x564f322eb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 -- 172.21.15.99:0/167723121 shutdown_connections 2024-03-23T23:45:33.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff68d800640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 0x7ff6700a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.848+0000 7ff692fbb740 1 -- 172.21.15.99:0/167723121 wait complete. 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.849+0000 7ff692fbb740 1 -- start start 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.849+0000 7ff692fbb740 1 Processor -- start 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.849+0000 7ff68d800640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 0x7ff6700a4ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x564f322ec910 tx=0x7ff67c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f32301630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f322f8b60 con 0x564f322f0a40 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f32210e80 con 0x564f322f5bb0 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f322f8ce0 con 0x564f322f0050 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff692fbb740 10 client.0 _ll_register_callbacks cb 0x564f32223130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff68d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff68ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff68d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41006/0 (socket says 172.21.15.99:41006) 2024-03-23T23:45:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff68d800640 1 -- 172.21.15.99:0/128617858 learned_addr learned my addr 172.21.15.99:0/128617858 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.850+0000 7ff68e200640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f32301630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 -- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 msgr2=0x564f32301630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f32301630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 -- 172.21.15.99:0/128617858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 msgr2=0x564f322fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 -- 172.21.15.99:0/128617858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f320337d0 con 0x564f322f0050 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff68d800640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff67c004040 tx=0x7ff67c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f8abed6fe288b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff66f400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00f810 con 0x564f322f0050 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff67c00f9b0 con 0x564f322f0050 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68000fc40 con 0x564f322f0050 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.851+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff67c00fbb0 con 0x564f322f0050 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 10 client.25972 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f31f693f0 con 0x564f322f0050 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff68000fc40 con 0x564f322f0050 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff67c026070 con 0x564f322f0050 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 10 client.25972 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 20 client.25972 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f3230a2b0 0x564f3230c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 4 client.25972 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f32262790 con 0x564f3230a2b0 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.852+0000 7ff68ce00640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f3230a2b0 0x564f3230c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.853+0000 7ff68ce00640 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f3230a2b0 0x564f3230c710 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fb4be5f07c566a server_cookie=e2768a359f010e65 in_seq=0 out_seq=0 2024-03-23T23:45:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.853+0000 7ff66f400640 10 client.25972 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.853+0000 7ff66f400640 1 -- 172.21.15.99:0/128617858 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f32262790 con 0x564f3230a2b0 2024-03-23T23:45:33.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.853+0000 7ff66f400640 10 client.25972 handle_command_reply: tid=0 2024-03-23T23:45:33.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 client.25972 shutdown 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 msgr2=0x564f322fa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff67c004040 tx=0x7ff67c004ab0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 shutdown_connections 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f3230a2b0 0x564f3230c710 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f32301630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f322fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 --2- 172.21.15.99:0/128617858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f322fa0b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 >> 172.21.15.99:0/128617858 conn(0x564f322b8200 msgr2=0x564f322eabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff66f400640 0 client.25972 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 shutdown_connections 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 1 -- 172.21.15.99:0/128617858 wait complete. 2024-03-23T23:45:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.854+0000 7ff692fbb740 20 client.25972 trim_cache size 0 max 16384 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 Processor -- start 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 -- start start 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a6650 con 0x564f322f0a40 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a7fd0 con 0x564f322f0050 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff692fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a6cb0 con 0x564f322f5bb0 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.882+0000 7ff68d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68e200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68e200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41010/0 (socket says 172.21.15.99:41010) 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38286/0 (socket says 172.21.15.99:38286) 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68e200640 1 -- 172.21.15.99:0/2697450825 learned_addr learned my addr 172.21.15.99:0/2697450825 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 -- 172.21.15.99:0/2697450825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 msgr2=0x564f322ea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 -- 172.21.15.99:0/2697450825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 msgr2=0x564f322ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 -- 172.21.15.99:0/2697450825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f31f290f0 con 0x564f322f0a40 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68e200640 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff68ce00640 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6740bcf50 tx=0x7ff6740f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83fadf0f86df88f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.883+0000 7ff67a000640 1 -- 172.21.15.99:0/2697450825 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67404b040 con 0x564f322f0a40 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff67a000640 1 -- 172.21.15.99:0/2697450825 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6740c1060 con 0x564f322f0a40 2024-03-23T23:45:33.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff67a000640 1 -- 172.21.15.99:0/2697450825 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740035f0 con 0x564f322f0a40 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 -- 172.21.15.99:0/2697450825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 msgr2=0x564f323015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6740bcf50 tx=0x7ff6740f0040 comp rx=0 tx=0).stop 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 -- 172.21.15.99:0/2697450825 shutdown_connections 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f5bb0 0x564f322ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 --2- 172.21.15.99:0/2697450825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f0050 0x564f322ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.884+0000 7ff692fbb740 1 -- 172.21.15.99:0/2697450825 >> 172.21.15.99:0/2697450825 conn(0x564f322b8200 msgr2=0x564f322ef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.885+0000 7ff692fbb740 1 -- 172.21.15.99:0/2697450825 shutdown_connections 2024-03-23T23:45:33.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.885+0000 7ff692fbb740 1 -- 172.21.15.99:0/2697450825 wait complete. 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.885+0000 7ff692fbb740 1 -- start start 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.885+0000 7ff692fbb740 1 Processor -- start 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f322ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a7970 con 0x564f322f0a40 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a7310 con 0x564f322f5bb0 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881a4810 con 0x564f322f0050 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff692fbb740 10 client.0 _ll_register_callbacks cb 0x7ff67c003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41018/0 (socket says 172.21.15.99:41018) 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 -- 172.21.15.99:0/3092689854 learned_addr learned my addr 172.21.15.99:0/3092689854 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68e200640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f322ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 -- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 msgr2=0x564f322ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f322ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 -- 172.21.15.99:0/3092689854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 msgr2=0x564f323015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68d800640 1 -- 172.21.15.99:0/3092689854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f32300000 con 0x564f322f0050 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.886+0000 7ff68e200640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f322ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff68d800640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff67c01e4b0 tx=0x7ff67c01e8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130e0f8bb40577ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff66f400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0048c0 con 0x564f322f0050 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6740c1060 con 0x564f322f0050 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff67c01ec60 con 0x564f322f0050 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff67c01ee60 con 0x564f322f0050 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff692fbb740 10 client.25978 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:33.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.887+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564f323667e0 con 0x564f322f0050 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6740c1060 con 0x564f322f0050 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff67c026040 con 0x564f322f0050 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 10 client.25978 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 20 client.25978 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f32367140 0x564f32369560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 4 client.25978 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f3225cae0 con 0x564f32367140 2024-03-23T23:45:33.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.888+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f32367140 0x564f32369560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff68ce00640 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f32367140 0x564f32369560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7904d96ff6aceec1 server_cookie=f6a5aef2146e3716 in_seq=0 out_seq=0 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff66f400640 10 client.25978 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff66f400640 1 -- 172.21.15.99:0/3092689854 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f3225cae0 con 0x564f32367140 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff66f400640 10 client.25978 handle_command_reply: tid=0 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 client.25978 shutdown 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 msgr2=0x564f323010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff67c01e4b0 tx=0x7ff67c01e8d0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 shutdown_connections 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x564f32367140 0x564f32369560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff66f400640 0 client.25978 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f322f5bb0 0x564f322ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f322f0a40 0x564f323015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 --2- 172.21.15.99:0/3092689854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f322f0050 0x564f323010a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 >> 172.21.15.99:0/3092689854 conn(0x564f322b8200 msgr2=0x564f322f12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 shutdown_connections 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 1 -- 172.21.15.99:0/3092689854 wait complete. 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.889+0000 7ff692fbb740 20 client.25978 trim_cache size 0 max 16384 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "e2da8d31-f594-4253-b3f3-1ac31001ca5d": { 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "e2da8d31-f594-4253-b3f3-1ac31001ca5d", 2024-03-23T23:45:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 msgr2=0x7ff6700a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 0x7ff6700a4ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x564f322ec910 tx=0x7ff67c009500 comp rx=0 tx=0).stop 2024-03-23T23:45:33.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 msgr2=0x7ff688197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff67402efd0 tx=0x7ff67402c660 comp rx=0 tx=0).stop 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 shutdown_connections 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff6700a2a20 0x7ff6700a4ee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff688107260 0x7ff68819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff688104930 0x7ff688197e20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 --2- 172.21.15.99:0/3907898596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff688102000 0x7ff6881978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 >> 172.21.15.99:0/3907898596 conn(0x7ff6880fba10 msgr2=0x7ff688102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.890+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 shutdown_connections 2024-03-23T23:45:33.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:45:33.891+0000 7ff68ec00640 1 -- 172.21.15.99:0/3907898596 wait complete. 2024-03-23T23:45:33.916 INFO:tasks.cephfs.filesystem:scrub status for tag:e2da8d31-f594-4253-b3f3-1ac31001ca5d - {'path': '/', 'tag': 'e2da8d31-f594-4253-b3f3-1ac31001ca5d', 'options': 'recursive,force'} 2024-03-23T23:45:34.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:34 smithi099 ceph-mon[31715]: pgmap v450: 97 pgs: 97 active+clean; 8.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 685 KiB/s wr, 9 op/s 2024-03-23T23:45:34.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:34 smithi149 ceph-mon[36083]: pgmap v450: 97 pgs: 97 active+clean; 8.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 685 KiB/s wr, 9 op/s 2024-03-23T23:45:34.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:34 smithi174 ceph-mon[37002]: pgmap v450: 97 pgs: 97 active+clean; 8.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 685 KiB/s wr, 9 op/s 2024-03-23T23:45:35.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:35 smithi099 ceph-mon[31715]: scrub queued for path: / 2024-03-23T23:45:35.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:35 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:35.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:35 smithi099 ceph-mon[31715]: scrub summary: active paths [/] 2024-03-23T23:45:35.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:35 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:35 smithi149 ceph-mon[36083]: scrub queued for path: / 2024-03-23T23:45:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:35 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:35 smithi149 ceph-mon[36083]: scrub summary: active paths [/] 2024-03-23T23:45:35.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:35 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:35.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:35 smithi174 ceph-mon[37002]: scrub queued for path: / 2024-03-23T23:45:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:35 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:35 smithi174 ceph-mon[37002]: scrub summary: active paths [/] 2024-03-23T23:45:35.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:35 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:45:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:36 smithi099 ceph-mon[31715]: pgmap v451: 97 pgs: 97 active+clean; 9.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 680 op/s 2024-03-23T23:45:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:36 smithi149 ceph-mon[36083]: pgmap v451: 97 pgs: 97 active+clean; 9.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 680 op/s 2024-03-23T23:45:36.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:36 smithi174 ceph-mon[37002]: pgmap v451: 97 pgs: 97 active+clean; 9.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 680 op/s 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.994+0000 7f8969c00640 1 -- 172.21.15.99:0/2300632112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8964102700 msgr2=0x7f8964102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.994+0000 7f8969c00640 1 --2- 172.21.15.99:0/2300632112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8964102700 0x7f8964102b80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f894c009900 tx=0x7f894c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 -- 172.21.15.99:0/2300632112 shutdown_connections 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 --2- 172.21.15.99:0/2300632112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89641030c0 0x7f8964109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 --2- 172.21.15.99:0/2300632112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8964102700 0x7f8964102b80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 --2- 172.21.15.99:0/2300632112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f8964101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 -- 172.21.15.99:0/2300632112 >> 172.21.15.99:0/2300632112 conn(0x7f89640fca80 msgr2=0x7f89640feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 -- 172.21.15.99:0/2300632112 shutdown_connections 2024-03-23T23:45:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.995+0000 7f8969c00640 1 -- 172.21.15.99:0/2300632112 wait complete. 2024-03-23T23:45:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.996+0000 7f8969c00640 1 Processor -- start 2024-03-23T23:45:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.996+0000 7f8969c00640 1 -- start start 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 0x7f8964197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 0x7f896419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896410d1f0 con 0x7f8964102700 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896410d070 con 0x7f8964101510 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896410d370 con 0x7f89641030c0 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8963e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8963e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38320/0 (socket says 172.21.15.99:38320) 2024-03-23T23:45:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8963e00640 1 -- 172.21.15.99:0/4237379957 learned_addr learned my addr 172.21.15.99:0/4237379957 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8969200640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 0x7f896419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.997+0000 7f8963400640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 0x7f8964197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 -- 172.21.15.99:0/4237379957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 msgr2=0x7f896419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 0x7f896419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 -- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 msgr2=0x7f8964197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 0x7f8964197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 -- 172.21.15.99:0/4237379957 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896419f3e0 con 0x7f8964101510 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8969200640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 0x7f896419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963400640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 0x7f8964197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8963e00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f895c00ebc0 tx=0x7f895c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8960c00640 1 -- 172.21.15.99:0/4237379957 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c003b70 con 0x7f8964101510 2024-03-23T23:45:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8960c00640 1 -- 172.21.15.99:0/4237379957 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895c003d10 con 0x7f8964101510 2024-03-23T23:45:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896419f6d0 con 0x7f8964101510 2024-03-23T23:45:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8960c00640 1 -- 172.21.15.99:0/4237379957 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c019980 con 0x7f8964101510 2024-03-23T23:45:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:36.998+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896419fbf0 con 0x7f8964101510 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 Processor -- start 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 -- start start 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f8960c00640 1 -- 172.21.15.99:0/4237379957 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f895c019b20 con 0x7f8964101510 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cf2e3b0 con 0x56159cf70f50 2024-03-23T23:45:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159ced0830 con 0x56159cf6bde0 2024-03-23T23:45:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f896dfa7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159ce795d0 con 0x56159cf6b3f0 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f8969200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f8963e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.003+0000 7f8969200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:41044/0 (socket says 172.21.15.99:41044) 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 -- 172.21.15.99:0/1230826501 learned_addr learned my addr 172.21.15.99:0/1230826501 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8963400640 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 -- 172.21.15.99:0/1230826501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 msgr2=0x56159cf708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 -- 172.21.15.99:0/1230826501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 msgr2=0x56159cf733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 -- 172.21.15.99:0/1230826501 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159ce912c0 con 0x56159cf6b3f0 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8963400640 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8969200640 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8954011540 tx=0x7f8954011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96422b694b59eeab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.004+0000 7f8959600640 1 -- 172.21.15.99:0/1230826501 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895400f5c0 con 0x56159cf6b3f0 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f8959600640 1 -- 172.21.15.99:0/1230826501 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895400fc40 con 0x56159cf6b3f0 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f8959600640 1 -- 172.21.15.99:0/1230826501 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954019d20 con 0x56159cf6b3f0 2024-03-23T23:45:37.006 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f8960c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 0x7f89480a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f896dfa7740 1 -- 172.21.15.99:0/1230826501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 msgr2=0x56159cf6b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8954011540 tx=0x7f8954011a00 comp rx=0 tx=0).stop 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f8960c00640 1 -- 172.21.15.99:0/4237379957 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f895c0cf2c0 con 0x7f8964101510 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.005+0000 7f8963e00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 0x7f89480a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 -- 172.21.15.99:0/1230826501 shutdown_connections 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1230826501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf6b810 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 -- 172.21.15.99:0/1230826501 >> 172.21.15.99:0/1230826501 conn(0x56159cf66960 msgr2=0x56159cf66d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 -- 172.21.15.99:0/1230826501 shutdown_connections 2024-03-23T23:45:37.011 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 -- 172.21.15.99:0/1230826501 wait complete. 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f8963e00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 0x7f89480a4cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f895c0985b0 tx=0x7f895c00cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.006+0000 7f896dfa7740 1 -- start start 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 Processor -- start 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 0x56159cf79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 0x56159cf7a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf81040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159ce8ca20 con 0x56159cf70f50 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cecc8d0 con 0x56159cf6b3f0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cf73f00 con 0x56159cf6bde0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.007+0000 7f896dfa7740 10 client.0 _ll_register_callbacks cb 0x56159cec91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8969200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 0x56159cf79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 0x56159cf7a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8969200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 0x56159cf79ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38352/0 (socket says 172.21.15.99:38352) 2024-03-23T23:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8969200640 1 -- 172.21.15.99:0/163775212 learned_addr learned my addr 172.21.15.99:0/163775212 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8969200640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf81040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 -- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 msgr2=0x56159cf79ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 0x56159cf79ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 -- 172.21.15.99:0/163775212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 msgr2=0x56159cf81040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf81040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 -- 172.21.15.99:0/163775212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159cbe5210 con 0x56159cf6bde0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8963400640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 0x56159cf7a000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f894c036040 tx=0x7f894c004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311cf46e8373e91e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.008+0000 7f8952000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c005d50 con 0x56159cf6bde0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f894c004ed0 con 0x56159cf6bde0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f895400fc40 con 0x56159cf6bde0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c050070 con 0x56159cf6bde0 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f896dfa7740 10 client.25990 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56159ccb1620 con 0x56159cf6bde0 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.009+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f895400fc40 con 0x56159cf6bde0 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f894c003850 con 0x56159cf6bde0 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 10 client.25990 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 10 client.25990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 20 client.25990 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cf85970 0x56159cf87d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 4 client.25990 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56159ceddd70 con 0x56159cf85970 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.010+0000 7f8969200640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cf85970 0x56159cf87d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.011+0000 7f8969200640 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cf85970 0x56159cf87d60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d34eccb06868c5 server_cookie=414a4af563d65363 in_seq=0 out_seq=0 2024-03-23T23:45:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.011+0000 7f8952000640 10 client.25990 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.011+0000 7f8952000640 1 -- 172.21.15.99:0/163775212 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56159ceddd70 con 0x56159cf85970 2024-03-23T23:45:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.011+0000 7f8952000640 10 client.25990 handle_command_reply: tid=0 2024-03-23T23:45:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.011+0000 7f896dfa7740 1 client.25990 shutdown 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 msgr2=0x56159cf7a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 0x56159cf7a000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f894c036040 tx=0x7f894c004970 comp rx=0 tx=0).stop 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 shutdown_connections 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cf85970 0x56159cf87d60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf70f50 0x56159cf81040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6bde0 0x56159cf7a000 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f8952000640 0 client.25990 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 --2- 172.21.15.99:0/163775212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6b3f0 0x56159cf79ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 >> 172.21.15.99:0/163775212 conn(0x56159cf66960 msgr2=0x56159cf65b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 shutdown_connections 2024-03-23T23:45:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 1 -- 172.21.15.99:0/163775212 wait complete. 2024-03-23T23:45:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.012+0000 7f896dfa7740 20 client.25990 trim_cache size 0 max 16384 2024-03-23T23:45:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 Processor -- start 2024-03-23T23:45:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 -- start start 2024-03-23T23:45:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6b3f0 0x56159cf78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf70f50 0x56159cf78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a6580 con 0x56159cf6b3f0 2024-03-23T23:45:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a7f00 con 0x56159cf6bde0 2024-03-23T23:45:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.029+0000 7f896dfa7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a6be0 con 0x56159cf70f50 2024-03-23T23:45:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8969200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6b3f0 0x56159cf78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:38356/0 (socket says 172.21.15.99:38356) 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 -- 172.21.15.99:0/1548475530 learned_addr learned my addr 172.21.15.99:0/1548475530 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8969200640 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf70f50 0x56159cf78660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 -- 172.21.15.99:0/1548475530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf70f50 msgr2=0x56159cf78660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf70f50 0x56159cf78660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 -- 172.21.15.99:0/1548475530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6b3f0 msgr2=0x56159cf78120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6b3f0 0x56159cf78120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 -- 172.21.15.99:0/1548475530 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159cf7aab0 con 0x56159cf6bde0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.030+0000 7f8963400640 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f894c003c70 tx=0x7f894c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be23e0dc5272a577 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f8958c00640 1 -- 172.21.15.99:0/1548475530 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c005d50 con 0x56159cf6bde0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f8958c00640 1 -- 172.21.15.99:0/1548475530 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f894c059070 con 0x56159cf6bde0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f8958c00640 1 -- 172.21.15.99:0/1548475530 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0053b0 con 0x56159cf6bde0 2024-03-23T23:45:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 -- 172.21.15.99:0/1548475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 msgr2=0x56159cf81000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f894c003c70 tx=0x7f894c005fd0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 -- 172.21.15.99:0/1548475530 shutdown_connections 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf70f50 0x56159cf78660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf6bde0 0x56159cf81000 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 --2- 172.21.15.99:0/1548475530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6b3f0 0x56159cf78120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 -- 172.21.15.99:0/1548475530 >> 172.21.15.99:0/1548475530 conn(0x56159cf66960 msgr2=0x56159cf74ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.031+0000 7f896dfa7740 1 -- 172.21.15.99:0/1548475530 shutdown_connections 2024-03-23T23:45:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.032+0000 7f896dfa7740 1 -- 172.21.15.99:0/1548475530 wait complete. 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.032+0000 7f896dfa7740 1 -- start start 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.032+0000 7f896dfa7740 1 Processor -- start 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf7e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf70f50 0x56159cf7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a78a0 con 0x56159cf6bde0 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a7240 con 0x56159cf70f50 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641a4740 con 0x56159cf6b3f0 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f896dfa7740 10 client.0 _ll_register_callbacks cb 0x7f895c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f8963400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f8969200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf7e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f8963400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57506/0 (socket says 172.21.15.99:57506) 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f8963400640 1 -- 172.21.15.99:0/527345100 learned_addr learned my addr 172.21.15.99:0/527345100 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.033+0000 7f8969200640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf70f50 0x56159cf7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 -- 172.21.15.99:0/527345100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 msgr2=0x56159cf7e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf7e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 -- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf70f50 msgr2=0x56159cf7fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf70f50 0x56159cf7fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 -- 172.21.15.99:0/527345100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159cf80050 con 0x56159cf6bde0 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8963400640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f894c004910 tx=0x7f894c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f67d95d784071dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.034+0000 7f8952000640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.036+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c005d50 con 0x56159cf6bde0 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.036+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f894c0050d0 con 0x56159cf6bde0 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.036+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f894c059070 con 0x56159cf6bde0 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.036+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c06b070 con 0x56159cf6bde0 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 10 client.15780 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56159cf81360 con 0x56159cf6bde0 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f894c059070 con 0x56159cf6bde0 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f894c04a500 con 0x56159cf6bde0 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 20 client.15780 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cfe17d0 0x56159cfe3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 4 client.15780 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56159ced80c0 con 0x56159cfe17d0 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.037+0000 7f8969200640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cfe17d0 0x56159cfe3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.038+0000 7f8952000640 10 client.15780 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:37.039 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.038+0000 7f8969200640 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cfe17d0 0x56159cfe3bf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ba2d0f44e203f2 server_cookie=ea3de7f2e39a27a7 in_seq=0 out_seq=0 2024-03-23T23:45:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f8952000640 1 -- 172.21.15.99:0/527345100 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56159ced80c0 con 0x56159cfe17d0 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f8952000640 10 client.15780 handle_command_reply: tid=0 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 client.15780 shutdown 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 msgr2=0x56159cf7ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f894c004910 tx=0x7f894c003820 comp rx=0 tx=0).stop 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 shutdown_connections 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56159cfe17d0 0x56159cfe3bf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56159cf70f50 0x56159cf7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56159cf6bde0 0x56159cf7ea50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 --2- 172.21.15.99:0/527345100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56159cf6b3f0 0x56159cf7e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f8952000640 0 client.15780 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.049+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 >> 172.21.15.99:0/527345100 conn(0x56159cf66960 msgr2=0x56159cf79990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.050+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 shutdown_connections 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.050+0000 7f896dfa7740 1 -- 172.21.15.99:0/527345100 wait complete. 2024-03-23T23:45:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.050+0000 7f896dfa7740 20 client.15780 trim_cache size 0 max 16384 2024-03-23T23:45:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:45:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:45:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:45:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.050+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 msgr2=0x7f89480a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.050+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 0x7f89480a4cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f895c0985b0 tx=0x7f895c00cf90 comp rx=0 tx=0).stop 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 msgr2=0x7f89641977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f895c00ebc0 tx=0x7f895c00c6a0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 shutdown_connections 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f89480a27f0 0x7f89480a4cb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89641030c0 0x7f896419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8964102700 0x7f8964197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 --2- 172.21.15.99:0/4237379957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8964101510 0x7f89641977c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 >> 172.21.15.99:0/4237379957 conn(0x7f89640fca80 msgr2=0x7f8964105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 shutdown_connections 2024-03-23T23:45:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:37.051+0000 7f8969c00640 1 -- 172.21.15.99:0/4237379957 wait complete. 2024-03-23T23:45:38.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:38 smithi099 ceph-mon[31715]: pgmap v452: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 682 op/s 2024-03-23T23:45:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:38 smithi149 ceph-mon[36083]: pgmap v452: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 682 op/s 2024-03-23T23:45:38.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:38 smithi174 ceph-mon[37002]: pgmap v452: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 682 op/s 2024-03-23T23:45:39.000 INFO:tasks.workunit.client.0.smithi099.stdout:.......... 2024-03-23T23:45:39.000 INFO:tasks.workunit.client.0.smithi099.stdout:do_o_direct_reads: SUCCESS 2024-03-23T23:45:39.006 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T23:45:39.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T23:45:39.102 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-03-23T23:45:39.102 DEBUG:teuthology.orchestra.run.smithi099:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-03-23T23:45:39.167 INFO:tasks.workunit.client.0.smithi099.stdout:test read from hole 2024-03-23T23:45:39.167 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo 'test read from hole' 2024-03-23T23:45:39.167 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-03-23T23:45:39.172 INFO:tasks.workunit.client.0.smithi099.stderr:0+0 records in 2024-03-23T23:45:39.172 INFO:tasks.workunit.client.0.smithi099.stderr:0+0 records out 2024-03-23T23:45:39.172 INFO:tasks.workunit.client.0.smithi099.stderr:0 bytes copied, 0.000143317 s, 0.0 kB/s 2024-03-23T23:45:39.172 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-03-23T23:45:39.177 INFO:tasks.workunit.client.0.smithi099.stderr:2+0 records in 2024-03-23T23:45:39.177 INFO:tasks.workunit.client.0.smithi099.stderr:2+0 records out 2024-03-23T23:45:39.177 INFO:tasks.workunit.client.0.smithi099.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00261652 s, 391 kB/s 2024-03-23T23:45:39.177 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-03-23T23:45:39.180 INFO:tasks.workunit.client.0.smithi099.stderr:2+0 records in 2024-03-23T23:45:39.180 INFO:tasks.workunit.client.0.smithi099.stderr:2+0 records out 2024-03-23T23:45:39.180 INFO:tasks.workunit.client.0.smithi099.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.9695e-05 s, 11.4 MB/s 2024-03-23T23:45:39.180 INFO:tasks.workunit.client.0.smithi099.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-03-23T23:45:39.182 INFO:tasks.workunit.client.0.smithi099.stdout:other thing 2024-03-23T23:45:39.182 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo 'other thing' 2024-03-23T23:45:39.182 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-03-23T23:45:39.184 INFO:tasks.workunit.client.0.smithi099.stderr:1+0 records in 2024-03-23T23:45:39.184 INFO:tasks.workunit.client.0.smithi099.stderr:1+0 records out 2024-03-23T23:45:39.184 INFO:tasks.workunit.client.0.smithi099.stderr:500 bytes copied, 9.2356e-05 s, 5.4 MB/s 2024-03-23T23:45:39.184 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-03-23T23:45:39.196 INFO:tasks.workunit.client.0.smithi099.stderr:0+1 records in 2024-03-23T23:45:39.196 INFO:tasks.workunit.client.0.smithi099.stderr:0+1 records out 2024-03-23T23:45:39.196 INFO:tasks.workunit.client.0.smithi099.stderr:500 bytes copied, 0.0061996 s, 80.7 kB/s 2024-03-23T23:45:39.196 INFO:tasks.workunit.client.0.smithi099.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-03-23T23:45:39.203 INFO:tasks.workunit.client.0.smithi099.stderr:0+1 records in 2024-03-23T23:45:39.203 INFO:tasks.workunit.client.0.smithi099.stderr:0+1 records out 2024-03-23T23:45:39.203 INFO:tasks.workunit.client.0.smithi099.stderr:500 bytes copied, 0.00469631 s, 106 kB/s 2024-03-23T23:45:39.204 INFO:tasks.workunit.client.0.smithi099.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-03-23T23:45:39.206 INFO:tasks.workunit.client.0.smithi099.stdout:OK 2024-03-23T23:45:39.207 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo OK 2024-03-23T23:45:39.207 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T23:45:39.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T23:45:39.293 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:39 smithi099 ceph-mon[31715]: scrub summary: idle 2024-03-23T23:45:39.295 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-03-23T23:45:39.295 DEBUG:teuthology.orchestra.run.smithi099:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-03-23T23:45:39.371 INFO:tasks.workunit.client.0.smithi099.stdout:writing first 3 bytes of 10k file 2024-03-23T23:45:39.371 INFO:tasks.workunit.client.0.smithi099.stdout:reading O_DIRECT 2024-03-23T23:45:39.371 INFO:tasks.workunit.client.0.smithi099.stdout:got 10000 2024-03-23T23:45:39.372 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T23:45:39.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T23:45:39.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:39 smithi149 ceph-mon[36083]: scrub summary: idle 2024-03-23T23:45:39.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:39 smithi174 ceph-mon[37002]: scrub summary: idle 2024-03-23T23:45:39.459 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-03-23T23:45:39.459 DEBUG:teuthology.orchestra.run.smithi099:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-03-23T23:45:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:40 smithi099 ceph-mon[31715]: pgmap v453: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.52k op/s 2024-03-23T23:45:40.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:40 smithi149 ceph-mon[36083]: pgmap v453: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.52k op/s 2024-03-23T23:45:40.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:40 smithi174 ceph-mon[37002]: pgmap v453: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.52k op/s 2024-03-23T23:45:42.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:42 smithi099 ceph-mon[31715]: pgmap v454: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.53k op/s 2024-03-23T23:45:42.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:42 smithi149 ceph-mon[36083]: pgmap v454: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.53k op/s 2024-03-23T23:45:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:42 smithi174 ceph-mon[37002]: pgmap v454: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.4 MiB/s wr, 1.53k op/s 2024-03-23T23:45:44.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:44 smithi099 ceph-mon[31715]: pgmap v455: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 724 KiB/s wr, 1.53k op/s 2024-03-23T23:45:44.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:44 smithi149 ceph-mon[36083]: pgmap v455: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 724 KiB/s wr, 1.53k op/s 2024-03-23T23:45:44.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:44 smithi174 ceph-mon[37002]: pgmap v455: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 724 KiB/s wr, 1.53k op/s 2024-03-23T23:45:45.665 INFO:tasks.workunit.client.0.smithi099.stdout:writing pattern 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-03-23T23:45:45.666 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-03-23T23:45:45.667 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:45.668 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:45.669 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:45.670 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:45.671 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:45.672 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:45.673 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:45.715 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:45.716 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:45.717 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:45.718 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:45.719 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:45.720 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:45.721 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:45.722 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:45.748 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:45.748 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:45.749 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:45.750 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:45.751 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:45.752 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:45.753 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:45.754 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:45.755 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:45.782 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:45.783 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:45.784 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:45.785 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:45.786 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-03-23T23:45:45.787 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-23T23:45:45.788 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:45.789 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:45.826 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:45.826 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:45.826 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:45.827 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:45.828 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:45.829 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:45.830 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:45.831 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:45.832 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:45.865 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:45.865 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:45.866 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:45.867 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:45.868 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:45.869 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:45.870 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:45.871 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:45.897 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:45.897 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:45.897 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:45.897 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:45.898 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:45.899 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:45.900 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:45.901 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:45.902 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:45.903 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:45.903 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:45.903 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:45.903 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:45.903 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:45.931 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:45.931 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:45.931 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:45.931 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:45.931 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:45.932 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:45.933 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-23T23:45:45.934 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-23T23:45:45.935 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:45.936 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:45.937 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:45.975 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:45.975 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:45.975 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:45.975 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:45.975 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:45.976 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:45.977 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:45.978 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:45.979 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:45.980 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:45.981 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:45.981 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:45.981 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:46.009 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:46.010 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:46.011 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:46.012 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:46.013 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:46.014 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:46.035 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:46.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:46.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:46.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:46.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:46.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:46.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:46.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:46.061 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:46.062 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-23T23:45:46.063 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:46.064 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:46.065 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:46.066 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:46.090 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:46.090 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:46.090 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:46.090 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:46.091 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:46.092 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:46.093 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:46.094 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:46.095 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:46.096 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:46.096 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:46.125 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:46.125 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:46.125 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:46.125 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:46.125 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:46.126 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:46.127 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:46.128 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:46.129 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-23T23:45:46.130 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-03-23T23:45:46.131 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-23T23:45:46.131 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-23T23:45:46.151 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-03-23T23:45:46.152 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-03-23T23:45:46.153 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-03-23T23:45:46.154 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-03-23T23:45:46.155 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-23T23:45:46.156 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-03-23T23:45:46.190 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-23T23:45:46.190 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-03-23T23:45:46.190 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-03-23T23:45:46.191 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-03-23T23:45:46.192 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-23T23:45:46.193 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-03-23T23:45:46.194 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-23T23:45:46.195 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-03-23T23:45:46.196 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-03-23T23:45:46.229 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-03-23T23:45:46.230 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-03-23T23:45:46.231 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-23T23:45:46.232 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-23T23:45:46.233 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-23T23:45:46.234 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-03-23T23:45:46.235 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-23T23:45:46.262 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-23T23:45:46.263 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-23T23:45:46.264 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-23T23:45:46.265 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-23T23:45:46.266 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-03-23T23:45:46.267 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-03-23T23:45:46.290 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-03-23T23:45:46.291 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-23T23:45:46.292 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-23T23:45:46.293 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-23T23:45:46.294 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-03-23T23:45:46.295 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-23T23:45:46.296 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-03-23T23:45:46.296 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-23T23:45:46.296 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-03-23T23:45:46.318 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-03-23T23:45:46.319 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-23T23:45:46.320 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-03-23T23:45:46.321 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-03-23T23:45:46.322 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-23T23:45:46.323 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-23T23:45:46.324 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-03-23T23:45:46.356 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-23T23:45:46.356 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-03-23T23:45:46.356 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-23T23:45:46.356 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-03-23T23:45:46.357 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-23T23:45:46.358 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-03-23T23:45:46.359 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-03-23T23:45:46.360 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-23T23:45:46.361 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-03-23T23:45:46.362 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-23T23:45:46.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:46 smithi099 ceph-mon[31715]: pgmap v456: 97 pgs: 97 active+clean; 763 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 724 KiB/s wr, 1.66k op/s 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-03-23T23:45:46.387 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-23T23:45:46.388 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-23T23:45:46.389 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-03-23T23:45:46.390 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-23T23:45:46.391 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-03-23T23:45:46.392 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-23T23:45:46.393 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:46 smithi149 ceph-mon[36083]: pgmap v456: 97 pgs: 97 active+clean; 763 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 724 KiB/s wr, 1.66k op/s 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-03-23T23:45:46.412 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-03-23T23:45:46.413 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-03-23T23:45:46.414 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-03-23T23:45:46.415 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-03-23T23:45:46.416 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-03-23T23:45:46.417 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-23T23:45:46.426 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:46 smithi174 ceph-mon[37002]: pgmap v456: 97 pgs: 97 active+clean; 763 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 724 KiB/s wr, 1.66k op/s 2024-03-23T23:45:46.453 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-23T23:45:46.454 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-03-23T23:45:46.455 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-23T23:45:46.456 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-03-23T23:45:46.457 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-03-23T23:45:46.458 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-23T23:45:46.459 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-03-23T23:45:46.498 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-23T23:45:46.498 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-03-23T23:45:46.498 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-03-23T23:45:46.499 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-03-23T23:45:46.500 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-03-23T23:45:46.501 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-03-23T23:45:46.502 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-23T23:45:46.503 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-23T23:45:46.504 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-23T23:45:46.525 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-23T23:45:46.526 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-03-23T23:45:46.527 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-23T23:45:46.528 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-03-23T23:45:46.529 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-03-23T23:45:46.530 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-03-23T23:45:46.546 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-03-23T23:45:46.547 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-23T23:45:46.548 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-03-23T23:45:46.549 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-03-23T23:45:46.550 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-03-23T23:45:46.551 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-23T23:45:46.552 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-03-23T23:45:46.552 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-23T23:45:46.552 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-03-23T23:45:46.552 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-03-23T23:45:46.581 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-03-23T23:45:46.582 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-23T23:45:46.583 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-03-23T23:45:46.584 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-03-23T23:45:46.585 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-03-23T23:45:46.586 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-23T23:45:46.587 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-03-23T23:45:46.587 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-23T23:45:46.587 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-23T23:45:46.618 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-03-23T23:45:46.619 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-23T23:45:46.620 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-23T23:45:46.621 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-03-23T23:45:46.622 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-03-23T23:45:46.623 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-23T23:45:46.624 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-03-23T23:45:46.624 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-23T23:45:46.624 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-03-23T23:45:46.650 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-03-23T23:45:46.651 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-03-23T23:45:46.652 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-23T23:45:46.653 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-23T23:45:46.654 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-23T23:45:46.655 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-03-23T23:45:47.036 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-03-23T23:45:47.037 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-23T23:45:47.038 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-23T23:45:47.039 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-03-23T23:45:47.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-23T23:45:47.040 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-03-23T23:45:47.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-23T23:45:47.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-03-23T23:45:47.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-23T23:45:47.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-03-23T23:45:47.041 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:47.042 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-23T23:45:47.043 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:48.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:48 smithi099 ceph-mon[31715]: pgmap v457: 97 pgs: 97 active+clean; 784 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 KiB/s wr, 996 op/s 2024-03-23T23:45:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:48 smithi149 ceph-mon[36083]: pgmap v457: 97 pgs: 97 active+clean; 784 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 KiB/s wr, 996 op/s 2024-03-23T23:45:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:48 smithi174 ceph-mon[37002]: pgmap v457: 97 pgs: 97 active+clean; 784 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 KiB/s wr, 996 op/s 2024-03-23T23:45:48.434 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:48.434 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:48.434 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-23T23:45:48.434 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:48.434 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:48.435 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:48.436 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:48.437 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:48.438 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-23T23:45:48.439 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:48.440 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:48.441 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:48.441 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-23T23:45:48.441 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:48.441 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:48.441 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:49.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-23T23:45:49.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:49.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:49.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:49.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:49.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:49.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-23T23:45:50.053 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:50.053 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:50.053 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-23T23:45:50.054 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:50.055 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:50.056 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:50.057 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:50.058 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:50.059 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:50.060 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 -- 172.21.15.99:0/1153702389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1480107d00 msgr2=0x7f1480108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 --2- 172.21.15.99:0/1153702389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1480107d00 0x7f1480108120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1470009900 tx=0x7f147002bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 -- 172.21.15.99:0/1153702389 shutdown_connections 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 --2- 172.21.15.99:0/1153702389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14801098b0 0x7f1480110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 --2- 172.21.15.99:0/1153702389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480108ef0 0x7f1480109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 --2- 172.21.15.99:0/1153702389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1480107d00 0x7f1480108120 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 -- 172.21.15.99:0/1153702389 >> 172.21.15.99:0/1153702389 conn(0x7f1480076cb0 msgr2=0x7f14800790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 -- 172.21.15.99:0/1153702389 shutdown_connections 2024-03-23T23:45:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.199+0000 7f1487000640 1 -- 172.21.15.99:0/1153702389 wait complete. 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.200+0000 7f1487000640 1 Processor -- start 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.200+0000 7f1487000640 1 -- start start 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 0x7f148019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 0x7f148019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801139d0 con 0x7f1480107d00 2024-03-23T23:45:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480113850 con 0x7f1480108ef0 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1487000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480113b50 con 0x7f14801098b0 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52864/0 (socket says 172.21.15.99:52864) 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 -- 172.21.15.99:0/3964928940 learned_addr learned my addr 172.21.15.99:0/3964928940 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f147fe00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 0x7f148019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1477e00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 0x7f148019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 -- 172.21.15.99:0/3964928940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 msgr2=0x7f148019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 0x7f148019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 -- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 msgr2=0x7f148019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 0x7f148019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.201+0000 7f1485000640 1 -- 172.21.15.99:0/3964928940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14801a5c30 con 0x7f14801098b0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f1477e00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 0x7f148019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f147fe00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 0x7f148019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f1485000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f147800eb10 tx=0x7f147800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f147d600640 1 -- 172.21.15.99:0/3964928940 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478004b20 con 0x7f14801098b0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f147d600640 1 -- 172.21.15.99:0/3964928940 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478004cc0 con 0x7f14801098b0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f147d600640 1 -- 172.21.15.99:0/3964928940 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478010760 con 0x7f14801098b0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.202+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14801a5f20 con 0x7f14801098b0 2024-03-23T23:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.203+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14801a6390 con 0x7f14801098b0 2024-03-23T23:45:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.206+0000 7f147d600640 1 -- 172.21.15.99:0/3964928940 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1478010900 con 0x7f14801098b0 2024-03-23T23:45:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.207+0000 7f1489d6d740 1 Processor -- start 2024-03-23T23:45:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.207+0000 7f1489d6d740 1 -- start start 2024-03-23T23:45:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.207+0000 7f147d600640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 0x7f14640a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f147d600640 1 -- 172.21.15.99:0/3964928940 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14780cf070 con 0x7f14801098b0 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1477e00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 0x7f14640a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x56483839a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x56483839f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483835d3b0 con 0x56483839abd0 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648382ff830 con 0x56483839fd40 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1489d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648382a85d0 con 0x56483839a1e0 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f147fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x56483839a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f147fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x56483839a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52868/0 (socket says 172.21.15.99:52868) 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f147fe00640 1 -- 172.21.15.99:0/1864032053 learned_addr learned my addr 172.21.15.99:0/1864032053 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.208+0000 7f1477e00640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x56483839f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 -- 172.21.15.99:0/1864032053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 msgr2=0x56483839a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x56483839a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 -- 172.21.15.99:0/1864032053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 msgr2=0x56483839f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x56483839f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1485000640 1 -- 172.21.15.99:0/1864032053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648382c02c0 con 0x56483839fd40 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1477e00640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x56483839f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.209+0000 7f1477e00640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 0x7f14640a4db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f14680058c0 tx=0x7f146800bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.210+0000 7f1485000640 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383a21d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14780ccec0 tx=0x7f1478004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a564fa5a7b70fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.210+0000 7f1475600640 1 -- 172.21.15.99:0/1864032053 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147809b9f0 con 0x56483839fd40 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.210+0000 7f1475600640 1 -- 172.21.15.99:0/1864032053 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14780b4070 con 0x56483839fd40 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.210+0000 7f1475600640 1 -- 172.21.15.99:0/1864032053 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780a3d00 con 0x56483839fd40 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 -- 172.21.15.99:0/1864032053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 msgr2=0x5648383a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383a21d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14780ccec0 tx=0x7f1478004f20 comp rx=0 tx=0).stop 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 -- 172.21.15.99:0/1864032053 shutdown_connections 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383a21d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x56483839f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 --2- 172.21.15.99:0/1864032053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x56483839a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.211+0000 7f1489d6d740 1 -- 172.21.15.99:0/1864032053 >> 172.21.15.99:0/1864032053 conn(0x564838395750 msgr2=0x564838395b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.212+0000 7f1489d6d740 1 -- 172.21.15.99:0/1864032053 shutdown_connections 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.212+0000 7f1489d6d740 1 -- 172.21.15.99:0/1864032053 wait complete. 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.212+0000 7f1489d6d740 1 -- start start 2024-03-23T23:45:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.213+0000 7f1489d6d740 1 Processor -- start 2024-03-23T23:45:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.213+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.213+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.213+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.213+0000 7f1489d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648382bba20 con 0x56483839abd0 2024-03-23T23:45:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1489d6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648382fb8d0 con 0x56483839fd40 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1489d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648383a2c90 con 0x56483839a1e0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1489d6d740 10 client.0 _ll_register_callbacks cb 0x5648382f81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f147fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f147fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:52882/0 (socket says 172.21.15.99:52882) 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f147fe00640 1 -- 172.21.15.99:0/3553772808 learned_addr learned my addr 172.21.15.99:0/3553772808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1485000640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 -- 172.21.15.99:0/3553772808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 msgr2=0x5648383a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 -- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 msgr2=0x5648383ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.214+0000 7f1477e00640 1 -- 172.21.15.99:0/3553772808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564838014210 con 0x56483839abd0 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f147fe00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:45:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f1477e00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a4810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56483839b640 tx=0x7f1468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb12d3b9b5cc2ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f146e000640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468012d50 con 0x56483839abd0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14780b4070 con 0x56483839abd0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.215+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1468004020 con 0x56483839abd0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.216+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146802f050 con 0x56483839abd0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.216+0000 7f1489d6d740 10 client.15792 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.216+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5648380e0620 con 0x56483839abd0 2024-03-23T23:45:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.216+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14780b4070 con 0x56483839abd0 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1468032020 con 0x56483839abd0 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 20 client.15792 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648383b01b0 0x5648383b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 4 client.15792 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56483830cd70 con 0x5648383b01b0 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.217+0000 7f147fe00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648383b01b0 0x5648383b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.218+0000 7f147fe00640 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648383b01b0 0x5648383b2610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa14c305ca09a54 server_cookie=99f6a66d38788da0 in_seq=0 out_seq=0 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.218+0000 7f146e000640 10 client.15792 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f146e000640 1 -- 172.21.15.99:0/3553772808 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56483830cd70 con 0x5648383b01b0 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f146e000640 10 client.15792 handle_command_reply: tid=0 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 client.15792 shutdown 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 msgr2=0x5648383a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a4810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56483839b640 tx=0x7f1468004730 comp rx=0 tx=0).stop 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 shutdown_connections 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5648383b01b0 0x5648383b2610 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a4810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3553772808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 >> 172.21.15.99:0/3553772808 conn(0x564838395750 msgr2=0x564838394950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.219+0000 7f146e000640 0 client.15792 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.220+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 shutdown_connections 2024-03-23T23:45:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.220+0000 7f1489d6d740 1 -- 172.21.15.99:0/3553772808 wait complete. 2024-03-23T23:45:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.220+0000 7f1489d6d740 20 client.15792 trim_cache size 0 max 16384 2024-03-23T23:45:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.241+0000 7f1489d6d740 1 Processor -- start 2024-03-23T23:45:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.241+0000 7f1489d6d740 1 -- start start 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839a1e0 0x5648383989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839abd0 0x5648383ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801af130 con 0x56483839a1e0 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801b0ab0 con 0x56483839fd40 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1489d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801af790 con 0x56483839abd0 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:33782/0 (socket says 172.21.15.99:33782) 2024-03-23T23:45:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 -- 172.21.15.99:0/708609464 learned_addr learned my addr 172.21.15.99:0/708609464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 -- 172.21.15.99:0/708609464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839abd0 msgr2=0x5648383ab810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:45:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839abd0 0x5648383ab810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 -- 172.21.15.99:0/708609464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839a1e0 msgr2=0x5648383989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839a1e0 0x5648383989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.242+0000 7f1485000640 1 -- 172.21.15.99:0/708609464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483838f9c0 con 0x56483839fd40 2024-03-23T23:45:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.243+0000 7f1485000640 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1478004f90 tx=0x7f147800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9ab0a9e121738e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.243+0000 7f1474c00640 1 -- 172.21.15.99:0/708609464 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780a4190 con 0x56483839fd40 2024-03-23T23:45:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.243+0000 7f1474c00640 1 -- 172.21.15.99:0/708609464 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478064020 con 0x56483839fd40 2024-03-23T23:45:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.243+0000 7f1474c00640 1 -- 172.21.15.99:0/708609464 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478098060 con 0x56483839fd40 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 -- 172.21.15.99:0/708609464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 msgr2=0x564838398ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1478004f90 tx=0x7f147800cf70 comp rx=0 tx=0).stop 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 -- 172.21.15.99:0/708609464 shutdown_connections 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x564838398ef0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839abd0 0x5648383ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 --2- 172.21.15.99:0/708609464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839a1e0 0x5648383989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 -- 172.21.15.99:0/708609464 >> 172.21.15.99:0/708609464 conn(0x564838395750 msgr2=0x5648383a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.244+0000 7f1489d6d740 1 -- 172.21.15.99:0/708609464 shutdown_connections 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.245+0000 7f1489d6d740 1 -- 172.21.15.99:0/708609464 wait complete. 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.245+0000 7f1489d6d740 1 -- start start 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.245+0000 7f1489d6d740 1 Processor -- start 2024-03-23T23:45:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801b0450 con 0x56483839abd0 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801afdf0 con 0x56483839fd40 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801ad2f0 con 0x56483839a1e0 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1489d6d740 10 client.0 _ll_register_callbacks cb 0x7f146800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1485000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1477e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1485000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:33786/0 (socket says 172.21.15.99:33786) 2024-03-23T23:45:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.246+0000 7f1485000640 1 -- 172.21.15.99:0/3817571562 learned_addr learned my addr 172.21.15.99:0/3817571562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 -- 172.21.15.99:0/3817571562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 msgr2=0x5648383a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 -- 172.21.15.99:0/3817571562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 msgr2=0x5648383a9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 -- 172.21.15.99:0/3817571562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648383a2dd0 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1485000640 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14780ccc00 tx=0x7f14780ba370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19826e099e0348b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f146e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147807f060 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14780a3f10 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.247+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1478064020 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147809b9f0 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 10 client.25976 fetch_fsmap learned FSMap version 27 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56483840bfa0 con 0x56483839fd40 2024-03-23T23:45:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1478064020 con 0x56483839fd40 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f14780977d0 con 0x56483839fd40 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 10 client.25976 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 20 client.25976 populate_metadata read hostname 'smithi099' 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56483840c310 0x56483840e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 4 client.25976 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.248+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648383070c0 con 0x56483840c310 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.249+0000 7f147fe00640 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56483840c310 0x56483840e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.249+0000 7f147fe00640 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56483840c310 0x56483840e700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add18a5bf861d214 server_cookie=8903ed249d7e9223 in_seq=0 out_seq=0 2024-03-23T23:45:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.249+0000 7f146e000640 10 client.25976 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f146e000640 1 -- 172.21.15.99:0/3817571562 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648383070c0 con 0x56483840c310 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f146e000640 10 client.25976 handle_command_reply: tid=0 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 client.25976 shutdown 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 msgr2=0x5648383aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14780ccc00 tx=0x7f14780ba370 comp rx=0 tx=0).stop 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 shutdown_connections 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56483840c310 0x56483840e700 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56483839fd40 0x5648383aab70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483839abd0 0x5648383a9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 --2- 172.21.15.99:0/3817571562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56483839a1e0 0x5648383a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f146e000640 0 client.25976 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 >> 172.21.15.99:0/3817571562 conn(0x564838395750 msgr2=0x5648383989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 shutdown_connections 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.302+0000 7f1489d6d740 1 -- 172.21.15.99:0/3817571562 wait complete. 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1489d6d740 20 client.25976 trim_cache size 0 max 16384 2024-03-23T23:45:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:45:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:45:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:45:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 msgr2=0x7f14640a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 0x7f14640a4db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f14680058c0 tx=0x7f146800bc30 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 msgr2=0x7f14801a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f147800eb10 tx=0x7f147800c2d0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 shutdown_connections 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f14640a28f0 0x7f14640a4db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14801098b0 0x7f14801a55c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1480108ef0 0x7f148019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 --2- 172.21.15.99:0/3964928940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1480107d00 0x7f148019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:45:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.303+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 >> 172.21.15.99:0/3964928940 conn(0x7f1480076cb0 msgr2=0x7f148010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:45:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.304+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 shutdown_connections 2024-03-23T23:45:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:45:50.304+0000 7f1487000640 1 -- 172.21.15.99:0/3964928940 wait complete. 2024-03-23T23:45:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:50 smithi099 ceph-mon[31715]: pgmap v458: 97 pgs: 97 active+clean; 41 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 357 KiB/s wr, 1.05k op/s 2024-03-23T23:45:50.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:50 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:45:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:50 smithi149 ceph-mon[36083]: pgmap v458: 97 pgs: 97 active+clean; 41 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 357 KiB/s wr, 1.05k op/s 2024-03-23T23:45:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:50 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:45:50.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:50 smithi174 ceph-mon[37002]: pgmap v458: 97 pgs: 97 active+clean; 41 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 357 KiB/s wr, 1.05k op/s 2024-03-23T23:45:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:50 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:45:51.164 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:51.164 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-23T23:45:51.164 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:51.164 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:51.164 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:51.165 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-23T23:45:51.166 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:51.167 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:51.168 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:51.169 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:51.170 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:51.170 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-23T23:45:51.170 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:51.170 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:52 smithi099 ceph-mon[31715]: pgmap v459: 97 pgs: 97 active+clean; 159 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 892 KiB/s wr, 360 op/s 2024-03-23T23:45:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:45:52.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:45:52.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:45:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:52 smithi149 ceph-mon[36083]: pgmap v459: 97 pgs: 97 active+clean; 159 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 892 KiB/s wr, 360 op/s 2024-03-23T23:45:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:45:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:45:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:45:52.413 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:52.413 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-23T23:45:52.413 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:52.413 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:52.413 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-23T23:45:52.414 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:52.415 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:52.416 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:52.417 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-23T23:45:52.418 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-23T23:45:52.419 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:52.420 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:52.421 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-23T23:45:52.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:52 smithi174 ceph-mon[37002]: pgmap v459: 97 pgs: 97 active+clean; 159 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 892 KiB/s wr, 360 op/s 2024-03-23T23:45:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:45:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:45:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:45:52.990 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:52.990 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:52.990 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:52.990 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:52.991 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-23T23:45:52.992 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:52.993 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:52.994 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-23T23:45:52.995 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:52.996 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:52.997 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:53.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-23T23:45:53.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:53.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:53.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:53.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-23T23:45:53.643 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:54.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:54 smithi099 ceph-mon[31715]: pgmap v460: 97 pgs: 97 active+clean; 223 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 924 KiB/s wr, 380 op/s 2024-03-23T23:45:54.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:54 smithi149 ceph-mon[36083]: pgmap v460: 97 pgs: 97 active+clean; 223 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 924 KiB/s wr, 380 op/s 2024-03-23T23:45:54.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:54 smithi174 ceph-mon[37002]: pgmap v460: 97 pgs: 97 active+clean; 223 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 924 KiB/s wr, 380 op/s 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:55.007 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:55.008 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:55.009 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-23T23:45:55.010 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:55.011 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-23T23:45:55.012 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:56.186 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-23T23:45:56.187 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:56.188 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:56.189 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-23T23:45:56.190 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:56.191 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:56.192 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:56.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:56 smithi099 ceph-mon[31715]: pgmap v461: 97 pgs: 97 active+clean; 215 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 968 KiB/s wr, 415 op/s 2024-03-23T23:45:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:56 smithi149 ceph-mon[36083]: pgmap v461: 97 pgs: 97 active+clean; 215 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 968 KiB/s wr, 415 op/s 2024-03-23T23:45:56.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:56 smithi174 ceph-mon[37002]: pgmap v461: 97 pgs: 97 active+clean; 215 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 968 KiB/s wr, 415 op/s 2024-03-23T23:45:56.736 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:56.736 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-23T23:45:56.736 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:56.736 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-23T23:45:56.737 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:56.738 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:56.739 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:56.740 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:56.741 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:56.742 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:57.636 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:57.637 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-23T23:45:57.638 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:57.639 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:57.640 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:57.641 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:57.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-23T23:45:57.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:57.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:57.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:57.642 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-23T23:45:58.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:45:58 smithi099 ceph-mon[31715]: pgmap v462: 97 pgs: 97 active+clean; 201 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-03-23T23:45:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:45:58 smithi149 ceph-mon[36083]: pgmap v462: 97 pgs: 97 active+clean; 201 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-03-23T23:45:58.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:45:58 smithi174 ceph-mon[37002]: pgmap v462: 97 pgs: 97 active+clean; 201 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:58.940 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-23T23:45:58.941 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-23T23:45:58.942 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:58.943 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-23T23:45:58.944 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:58.945 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-23T23:45:58.946 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:58.946 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:58.946 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:58.946 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-23T23:45:58.946 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-23T23:45:59.808 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:59.809 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:59.810 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-23T23:45:59.811 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:59.812 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:59.813 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:59.814 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:59.814 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-23T23:45:59.814 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-23T23:46:00.352 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-03-23T23:46:00.353 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-03-23T23:46:00.354 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-03-23T23:46:00.355 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-23T23:46:00.356 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-03-23T23:46:00.357 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-23T23:46:00.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:00 smithi099 ceph-mon[31715]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 1.4 MiB/s wr, 428 op/s 2024-03-23T23:46:00.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:00 smithi149 ceph-mon[36083]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 1.4 MiB/s wr, 428 op/s 2024-03-23T23:46:00.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:00 smithi174 ceph-mon[37002]: pgmap v463: 97 pgs: 97 active+clean; 214 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 1.4 MiB/s wr, 428 op/s 2024-03-23T23:46:01.454 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-03-23T23:46:01.454 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-03-23T23:46:01.455 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-03-23T23:46:01.456 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-03-23T23:46:01.457 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-23T23:46:01.458 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-03-23T23:46:01.459 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-23T23:46:01.460 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-03-23T23:46:01.461 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-23T23:46:01.461 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-03-23T23:46:02.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:02 smithi099 ceph-mon[31715]: pgmap v464: 97 pgs: 97 active+clean; 162 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-03-23T23:46:02.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:02 smithi149 ceph-mon[36083]: pgmap v464: 97 pgs: 97 active+clean; 162 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-03-23T23:46:02.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:02 smithi174 ceph-mon[37002]: pgmap v464: 97 pgs: 97 active+clean; 162 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-03-23T23:46:02.963 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-03-23T23:46:02.964 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-23T23:46:02.965 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-03-23T23:46:02.966 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-03-23T23:46:02.967 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-03-23T23:46:02.968 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-23T23:46:03.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:03 smithi149 ceph-mon[36083]: pgmap v465: 97 pgs: 97 active+clean; 106 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 638 KiB/s wr, 287 op/s 2024-03-23T23:46:03.426 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:03 smithi174 ceph-mon[37002]: pgmap v465: 97 pgs: 97 active+clean; 106 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 638 KiB/s wr, 287 op/s 2024-03-23T23:46:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 -- 172.21.15.99:0/3051550992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44074080 msgr2=0x7f1a440744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 --2- 172.21.15.99:0/3051550992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44074080 0x7f1a440744e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009900 tx=0x7f1a2c02bcd0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 -- 172.21.15.99:0/3051550992 shutdown_connections 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 --2- 172.21.15.99:0/3051550992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a441093c0 0x7f1a4410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 --2- 172.21.15.99:0/3051550992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44074080 0x7f1a440744e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 --2- 172.21.15.99:0/3051550992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44075660 0x7f1a44073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 -- 172.21.15.99:0/3051550992 >> 172.21.15.99:0/3051550992 conn(0x7f1a440fca80 msgr2=0x7f1a440feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 -- 172.21.15.99:0/3051550992 shutdown_connections 2024-03-23T23:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.446+0000 7f1a49c00640 1 -- 172.21.15.99:0/3051550992 wait complete. 2024-03-23T23:46:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.448+0000 7f1a49c00640 1 Processor -- start 2024-03-23T23:46:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.448+0000 7f1a49c00640 1 -- start start 2024-03-23T23:46:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 0x7f1a44199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44075660 0x7f1a44199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4410f390 con 0x7f1a44075660 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4410f210 con 0x7f1a441093c0 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4410f510 con 0x7f1a44074080 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47818/0 (socket says 172.21.15.99:47818) 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49200640 1 -- 172.21.15.99:0/1035757089 learned_addr learned my addr 172.21.15.99:0/1035757089 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a43e00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 0x7f1a44199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a3be00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44075660 0x7f1a44199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.449+0000 7f1a49200640 1 -- 172.21.15.99:0/1035757089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 msgr2=0x7f1a44199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49200640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 0x7f1a44199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49200640 1 -- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44075660 msgr2=0x7f1a44199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49200640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44075660 0x7f1a44199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49200640 1 -- 172.21.15.99:0/1035757089 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a441a1640 con 0x7f1a441093c0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a43e00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 0x7f1a44199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49200640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9b0 tx=0x7f1a3c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a41600640 1 -- 172.21.15.99:0/1035757089 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00cd80 con 0x7f1a441093c0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a441a1930 con 0x7f1a441093c0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a41600640 1 -- 172.21.15.99:0/1035757089 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c0035f0 con 0x7f1a441093c0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a41600640 1 -- 172.21.15.99:0/1035757089 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c010670 con 0x7f1a441093c0 2024-03-23T23:46:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.450+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a441a1e00 con 0x7f1a441093c0 2024-03-23T23:46:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a4dfbe740 1 Processor -- start 2024-03-23T23:46:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a41600640 1 -- 172.21.15.99:0/1035757089 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1a3c0037a0 con 0x7f1a441093c0 2024-03-23T23:46:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a4dfbe740 1 -- start start 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc5b3f0 0x56166fc5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.455+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc60f50 0x56166fc633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc1e3b0 con 0x56166fc60f50 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbc0830 con 0x56166fc5b3f0 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fb695d0 con 0x56166fc5bde0 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50472/0 (socket says 172.21.15.99:50472) 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 -- 172.21.15.99:0/410291599 learned_addr learned my addr 172.21.15.99:0/410291599 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a3be00640 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc5b3f0 0x56166fc5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a3be00640 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc60f50 0x56166fc633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 -- 172.21.15.99:0/410291599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc5b3f0 msgr2=0x56166fc5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc5b3f0 0x56166fc5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 -- 172.21.15.99:0/410291599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc60f50 msgr2=0x56166fc633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc60f50 0x56166fc633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.456+0000 7f1a43e00640 1 -- 172.21.15.99:0/410291599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166fb812c0 con 0x56166fc5bde0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a43e00640 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a340104a0 tx=0x7f1a34010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26d7222950d69af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a41600640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 0x7f1a280a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a39600640 1 -- 172.21.15.99:0/410291599 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017070 con 0x56166fc5bde0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a41600640 1 -- 172.21.15.99:0/1035757089 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a3c0ceeb0 con 0x7f1a441093c0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a39600640 1 -- 172.21.15.99:0/410291599 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a34011be0 con 0x56166fc5bde0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a39600640 1 -- 172.21.15.99:0/410291599 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400edf0 con 0x56166fc5bde0 2024-03-23T23:46:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.457+0000 7f1a3be00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 0x7f1a280a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/410291599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 msgr2=0x56166fc608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a340104a0 tx=0x7f1a34010960 comp rx=0 tx=0).stop 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a3be00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 0x7f1a280a4cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/410291599 shutdown_connections 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc60f50 0x56166fc633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc608e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/410291599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc5b3f0 0x56166fc5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/410291599 >> 172.21.15.99:0/410291599 conn(0x56166fc56960 msgr2=0x56166fc56d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/410291599 shutdown_connections 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/410291599 wait complete. 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.458+0000 7f1a4dfbe740 1 -- start start 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 Processor -- start 2024-03-23T23:46:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5b3f0 0x56166fc655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fb7ca20 con 0x56166fc5b3f0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbbc8d0 con 0x56166fc60f50 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc63f00 con 0x56166fc5bde0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a4dfbe740 10 client.0 _ll_register_callbacks cb 0x56166fbb91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50476/0 (socket says 172.21.15.99:50476) 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47834/0 (socket says 172.21.15.99:47834) 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a3be00640 1 -- 172.21.15.99:0/2840430325 learned_addr learned my addr 172.21.15.99:0/2840430325 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.459+0000 7f1a43e00640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5b3f0 0x56166fc655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 -- 172.21.15.99:0/2840430325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 msgr2=0x56166fc65ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 -- 172.21.15.99:0/2840430325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5b3f0 msgr2=0x56166fc655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5b3f0 0x56166fc655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 -- 172.21.15.99:0/2840430325 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56166f8d5210 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a3be00640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a49200640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c09b530 tx=0x7f1a3c09c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71de539caf0efdc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a32000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b3020 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a3c003b10 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.460+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a34011be0 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0a3b40 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 10 client.25988 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56166f9a1620 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a34011be0 con 0x56166fc60f50 2024-03-23T23:46:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1a3c09ac80 con 0x56166fc60f50 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 10 client.25988 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 20 client.25988 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fc713c0 0x56166fc737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 4 client.25988 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.461+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56166fbcdd70 con 0x56166fc713c0 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.462+0000 7f1a43e00640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fc713c0 0x56166fc737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.462+0000 7f1a43e00640 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fc713c0 0x56166fc737b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ae76264ce43fdb server_cookie=f6ca7561f09ae39c in_seq=0 out_seq=0 2024-03-23T23:46:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.462+0000 7f1a32000640 10 client.25988 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.463+0000 7f1a32000640 1 -- 172.21.15.99:0/2840430325 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56166fbcdd70 con 0x56166fc713c0 2024-03-23T23:46:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.463+0000 7f1a32000640 10 client.25988 handle_command_reply: tid=0 2024-03-23T23:46:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.463+0000 7f1a4dfbe740 1 client.25988 shutdown 2024-03-23T23:46:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 msgr2=0x56166fc6cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c09b530 tx=0x7f1a3c09c070 comp rx=0 tx=0).stop 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 shutdown_connections 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fc713c0 0x56166fc737b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6cb20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a32000640 0 client.25988 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5bde0 0x56166fc65ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/2840430325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5b3f0 0x56166fc655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 >> 172.21.15.99:0/2840430325 conn(0x56166fc56960 msgr2=0x56166fc55b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.464+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 shutdown_connections 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.465+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/2840430325 wait complete. 2024-03-23T23:46:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.465+0000 7f1a4dfbe740 20 client.25988 trim_cache size 0 max 16384 2024-03-23T23:46:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.483+0000 7f1a4dfbe740 1 Processor -- start 2024-03-23T23:46:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.483+0000 7f1a4dfbe740 1 -- start start 2024-03-23T23:46:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441aaa00 con 0x56166fc5bde0 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441ac380 con 0x56166fc60f50 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441ab060 con 0x56166fc5b3f0 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47850/0 (socket says 172.21.15.99:47850) 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 -- 172.21.15.99:0/774245190 learned_addr learned my addr 172.21.15.99:0/774245190 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 -- 172.21.15.99:0/774245190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 msgr2=0x56166fc516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 -- 172.21.15.99:0/774245190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 msgr2=0x56166fc6cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.484+0000 7f1a49200640 1 -- 172.21.15.99:0/774245190 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166fc64040 con 0x56166fc60f50 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.485+0000 7f1a49200640 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0cc9e0 tx=0x7f1a3c09fac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4942fef6760e4af server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.485+0000 7f1a38c00640 1 -- 172.21.15.99:0/774245190 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b3030 con 0x56166fc60f50 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.485+0000 7f1a38c00640 1 -- 172.21.15.99:0/774245190 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c0cce50 con 0x56166fc60f50 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.485+0000 7f1a38c00640 1 -- 172.21.15.99:0/774245190 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c003c10 con 0x56166fc60f50 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/774245190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 msgr2=0x56166fc5a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0cc9e0 tx=0x7f1a3c09fac0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/774245190 shutdown_connections 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc5a2d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/774245190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/774245190 >> 172.21.15.99:0/774245190 conn(0x56166fc56960 msgr2=0x56166fc59bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/774245190 shutdown_connections 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/774245190 wait complete. 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 -- start start 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.486+0000 7f1a4dfbe740 1 Processor -- start 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441abd20 con 0x56166fc5bde0 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441ab6c0 con 0x56166fc60f50 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441a8bc0 con 0x56166fc5b3f0 2024-03-23T23:46:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a4dfbe740 10 client.0 _ll_register_callbacks cb 0x7f1a3c0973b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50480/0 (socket says 172.21.15.99:50480) 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 -- 172.21.15.99:0/3156539206 learned_addr learned my addr 172.21.15.99:0/3156539206 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a49200640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a3be00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 -- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 msgr2=0x56166fc6a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 -- 172.21.15.99:0/3156539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 msgr2=0x56166fc6c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a43e00640 1 -- 172.21.15.99:0/3156539206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56166fc6af00 con 0x56166fc5b3f0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.487+0000 7f1a3be00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a43e00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1a340104a0 tx=0x7f1a34004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ee309265b8ce32 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017040 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a3402b250 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a3c0cce50 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a3402b820 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a4dfbe740 10 client.26026 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56166fccd5d0 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c0cce50 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.488+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1a34011cd0 con 0x56166fc5b3f0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 10 client.26026 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 20 client.26026 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fccdcc0 0x56166fcd00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 4 client.26026 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56166fbc80c0 con 0x56166fccdcc0 2024-03-23T23:46:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.489+0000 7f1a3be00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fccdcc0 0x56166fcd00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.490+0000 7f1a32000640 10 client.26026 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.490+0000 7f1a3be00640 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fccdcc0 0x56166fcd00e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f7c1f16bdf0ab5 server_cookie=eff589798256b1e5 in_seq=0 out_seq=0 2024-03-23T23:46:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.559+0000 7f1a32000640 1 -- 172.21.15.99:0/3156539206 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56166fbc80c0 con 0x56166fccdcc0 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.559+0000 7f1a32000640 10 client.26026 handle_command_reply: tid=0 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 client.26026 shutdown 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 msgr2=0x56166fc6c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1a340104a0 tx=0x7f1a34004a30 comp rx=0 tx=0).stop 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 shutdown_connections 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x56166fccdcc0 0x56166fcd00e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56166fc60f50 0x56166fc6a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56166fc5bde0 0x56166fc6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a32000640 0 client.26026 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 --2- 172.21.15.99:0/3156539206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56166fc5b3f0 0x56166fc6c2b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 >> 172.21.15.99:0/3156539206 conn(0x56166fc56960 msgr2=0x56166fc5a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 shutdown_connections 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 1 -- 172.21.15.99:0/3156539206 wait complete. 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.560+0000 7f1a4dfbe740 20 client.26026 trim_cache size 0 max 16384 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-23T23:46:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 msgr2=0x7f1a280a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 0x7f1a280a4cb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c02e040 comp rx=0 tx=0).stop 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 msgr2=0x7f1a441a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9b0 tx=0x7f1a3c00ee70 comp rx=0 tx=0).stop 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 shutdown_connections 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f1a280a27f0 0x7f1a280a4cb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a441093c0 0x7f1a441a0fd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44075660 0x7f1a44199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 --2- 172.21.15.99:0/1035757089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44074080 0x7f1a44199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 >> 172.21.15.99:0/1035757089 conn(0x7f1a440fca80 msgr2=0x7f1a44105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 shutdown_connections 2024-03-23T23:46:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-23T23:46:03.561+0000 7f1a49c00640 1 -- 172.21.15.99:0/1035757089 wait complete. 2024-03-23T23:46:03.574 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-03-23T23:46:03.575 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-23T23:46:03.576 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-23T23:46:03.577 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-03-23T23:46:03.578 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-23T23:46:03.578 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-03-23T23:46:03.578 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-23T23:46:03.578 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-03-23T23:46:03.578 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-23T23:46:03.579 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-23T23:46:03.580 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-03-23T23:46:03.581 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-23T23:46:03.581 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-03-23T23:46:03.581 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-23T23:46:03.611 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:03 smithi099 ceph-mon[31715]: pgmap v465: 97 pgs: 97 active+clean; 106 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 638 KiB/s wr, 287 op/s 2024-03-23T23:46:03.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:46:03.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:46:04.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 -- 172.21.15.99:0/2248821619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8104ad0 msgr2=0x7f05c8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 --2- 172.21.15.99:0/2248821619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8104ad0 0x7f05c8104f50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009860 tx=0x7f05b402bb70 comp rx=0 tx=0).stop 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 -- 172.21.15.99:0/2248821619 shutdown_connections 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 --2- 172.21.15.99:0/2248821619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c8105490 0x7f05c810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 --2- 172.21.15.99:0/2248821619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8104ad0 0x7f05c8104f50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 --2- 172.21.15.99:0/2248821619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c81038e0 0x7f05c8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.156+0000 7f05cf600640 1 -- 172.21.15.99:0/2248821619 >> 172.21.15.99:0/2248821619 conn(0x7f05c80fee50 msgr2=0x7f05c8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.157+0000 7f05cf600640 1 -- 172.21.15.99:0/2248821619 shutdown_connections 2024-03-23T23:46:04.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.157+0000 7f05cf600640 1 -- 172.21.15.99:0/2248821619 wait complete. 2024-03-23T23:46:04.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 Processor -- start 2024-03-23T23:46:04.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 -- start start 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 0x7f05c8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 0x7f05c81a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c810f540 con 0x7f05c81038e0 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c810f3c0 con 0x7f05c8104ad0 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cf600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c810f6c0 con 0x7f05c8105490 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05ccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 0x7f05c8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05cd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 0x7f05c81a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47882/0 (socket says 172.21.15.99:47882) 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.158+0000 7f05c7e00640 1 -- 172.21.15.99:0/3065743448 learned_addr learned my addr 172.21.15.99:0/3065743448 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 -- 172.21.15.99:0/3065743448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 msgr2=0x7f05c81a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 0x7f05c81a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 -- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 msgr2=0x7f05c8199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 0x7f05c8199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 -- 172.21.15.99:0/3065743448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c81a17f0 con 0x7f05c8104ad0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05cd600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 0x7f05c81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 0x7f05c8199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.159+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05b40349e0 tx=0x7f05b4032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.160+0000 7f05c5600640 1 -- 172.21.15.99:0/3065743448 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4041030 con 0x7f05c8104ad0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.160+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c81a1a80 con 0x7f05c8104ad0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.160+0000 7f05c5600640 1 -- 172.21.15.99:0/3065743448 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b402c510 con 0x7f05c8104ad0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.160+0000 7f05c5600640 1 -- 172.21.15.99:0/3065743448 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4039040 con 0x7f05c8104ad0 2024-03-23T23:46:04.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.160+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c81a1f70 con 0x7f05c8104ad0 2024-03-23T23:46:04.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 Processor -- start 2024-03-23T23:46:04.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05c5600640 1 -- 172.21.15.99:0/3065743448 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f05b4044070 con 0x7f05c8104ad0 2024-03-23T23:46:04.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 -- start start 2024-03-23T23:46:04.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d837370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 0x55bd9d839e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d797250 con 0x55bd9d8379e0 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d73fa30 con 0x55bd9d831e80 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.163+0000 7f05d2237740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d7932f0 con 0x55bd9d832870 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47886/0 (socket says 172.21.15.99:47886) 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 -- 172.21.15.99:0/730379107 learned_addr learned my addr 172.21.15.99:0/730379107 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05ccc00640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d837370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 -- 172.21.15.99:0/730379107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 msgr2=0x55bd9d837370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d837370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05c7e00640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 0x55bd9d839e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 -- 172.21.15.99:0/730379107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 msgr2=0x55bd9d839e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 0x55bd9d839e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05cd600640 1 -- 172.21.15.99:0/730379107 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9d72a690 con 0x55bd9d831e80 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.164+0000 7f05c7e00640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 0x55bd9d839e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.165+0000 7f05cd600640 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011810 tx=0x7f05c0011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6048aa94844cf76f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.165+0000 7f05b9c00640 1 -- 172.21.15.99:0/730379107 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0011310 con 0x55bd9d831e80 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.165+0000 7f05b9c00640 1 -- 172.21.15.99:0/730379107 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05c000f9a0 con 0x55bd9d831e80 2024-03-23T23:46:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.165+0000 7f05b9c00640 1 -- 172.21.15.99:0/730379107 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0005730 con 0x55bd9d831e80 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 -- 172.21.15.99:0/730379107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 msgr2=0x55bd9d8322a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011810 tx=0x7f05c0011cd0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05c5600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 0x7f05b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 -- 172.21.15.99:0/730379107 shutdown_connections 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d8379e0 0x55bd9d839e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d837370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05c5600640 1 -- 172.21.15.99:0/3065743448 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f05b40efff0 con 0x7f05c8104ad0 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 --2- 172.21.15.99:0/730379107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d8322a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 -- 172.21.15.99:0/730379107 >> 172.21.15.99:0/730379107 conn(0x55bd9d7fa200 msgr2=0x55bd9d82d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 0x7f05b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 -- 172.21.15.99:0/730379107 shutdown_connections 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.166+0000 7f05d2237740 1 -- 172.21.15.99:0/730379107 wait complete. 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.167+0000 7f05d2237740 1 -- start start 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.167+0000 7f05d2237740 1 Processor -- start 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.167+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 0x7f05b00a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55bd9d82e740 tx=0x7f05bc014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.167+0000 7f05d2237740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 0x55bd9d83be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.167+0000 7f05d2237740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d83c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05d2237740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d843400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05d2237740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d83a990 con 0x55bd9d832870 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05d2237740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d752e80 con 0x55bd9d8379e0 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05d2237740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9d83ab10 con 0x55bd9d831e80 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05d2237740 10 client.0 _ll_register_callbacks cb 0x55bd9d765130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d843400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d843400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47898/0 (socket says 172.21.15.99:47898) 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05cd600640 1 -- 172.21.15.99:0/1912937936 learned_addr learned my addr 172.21.15.99:0/1912937936 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 0x55bd9d83be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 -- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 msgr2=0x55bd9d843400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d843400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 -- 172.21.15.99:0/1912937936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 msgr2=0x55bd9d83c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d83c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.168+0000 7f05ccc00640 1 -- 172.21.15.99:0/1912937936 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9d5757d0 con 0x55bd9d831e80 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05ccc00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 0x55bd9d83be80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55bd9d832c70 tx=0x7f05bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4552620531fa3226 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05ae000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00fa30 con 0x55bd9d831e80 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f05bc00fbd0 con 0x55bd9d831e80 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05c000f9a0 con 0x55bd9d831e80 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05bc02d070 con 0x55bd9d831e80 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05d2237740 10 client.26038 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:04.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.169+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bd9d4ab3f0 con 0x55bd9d831e80 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05c000f9a0 con 0x55bd9d831e80 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f05bc01e480 con 0x55bd9d831e80 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 10 client.26038 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 10 client.26038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 20 client.26038 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d847d50 0x55bd9d84a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 4 client.26038 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd9d7a4790 con 0x55bd9d847d50 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.170+0000 7f05c7e00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d847d50 0x55bd9d84a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.171+0000 7f05c7e00640 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d847d50 0x55bd9d84a1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0711916c08a75c4 server_cookie=3e9c5cd91cb6e6e5 in_seq=0 out_seq=0 2024-03-23T23:46:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.171+0000 7f05ae000640 10 client.26038 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-03-23T23:46:04.177 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-03-23T23:46:04.178 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-23T23:46:04.179 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-03-23T23:46:04.180 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-23T23:46:04.181 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-03-23T23:46:04.182 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-23T23:46:04.183 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-03-23T23:46:04.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05ae000640 1 -- 172.21.15.99:0/1912937936 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bd9d7a4790 con 0x55bd9d847d50 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05ae000640 10 client.26038 handle_command_reply: tid=0 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 client.26038 shutdown 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 msgr2=0x55bd9d83be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 0x55bd9d83be80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55bd9d832c70 tx=0x7f05bc004730 comp rx=0 tx=0).stop 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 shutdown_connections 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d847d50 0x55bd9d84a1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d843400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d83c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 --2- 172.21.15.99:0/1912937936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d831e80 0x55bd9d83be80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 >> 172.21.15.99:0/1912937936 conn(0x55bd9d7fa200 msgr2=0x55bd9d82ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.173+0000 7f05ae000640 0 client.26038 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.174+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 shutdown_connections 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.174+0000 7f05d2237740 1 -- 172.21.15.99:0/1912937936 wait complete. 2024-03-23T23:46:04.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.174+0000 7f05d2237740 20 client.26038 trim_cache size 0 max 16384 2024-03-23T23:46:04.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.199+0000 7f05d2237740 1 Processor -- start 2024-03-23T23:46:04.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.199+0000 7f05d2237740 1 -- start start 2024-03-23T23:46:04.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 0x55bd9d833980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d8433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81aabc0 con 0x55bd9d831e80 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81ac540 con 0x55bd9d8379e0 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05d2237740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81ab220 con 0x55bd9d832870 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05ccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 0x55bd9d833980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d8433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05ccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 0x55bd9d833980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37818/0 (socket says 172.21.15.99:37818) 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47908/0 (socket says 172.21.15.99:47908) 2024-03-23T23:46:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.200+0000 7f05ccc00640 1 -- 172.21.15.99:0/3694408878 learned_addr learned my addr 172.21.15.99:0/3694408878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 -- 172.21.15.99:0/3694408878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 msgr2=0x55bd9d830a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 -- 172.21.15.99:0/3694408878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 msgr2=0x55bd9d833980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 0x55bd9d833980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 -- 172.21.15.99:0/3694408878 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9d46b0f0 con 0x55bd9d832870 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05cd600640 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d8433c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55bd9d83d360 tx=0x7f05b4040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d9b2963953b9b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05b9200640 1 -- 172.21.15.99:0/3694408878 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b40bc930 con 0x55bd9d832870 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05b9200640 1 -- 172.21.15.99:0/3694408878 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b40c0a80 con 0x55bd9d832870 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.201+0000 7f05b9200640 1 -- 172.21.15.99:0/3694408878 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b402c6c0 con 0x55bd9d832870 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 -- 172.21.15.99:0/3694408878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 msgr2=0x55bd9d8433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d8433c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55bd9d83d360 tx=0x7f05b4040000 comp rx=0 tx=0).stop 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 -- 172.21.15.99:0/3694408878 shutdown_connections 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d8379e0 0x55bd9d830a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d832870 0x55bd9d8433c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 --2- 172.21.15.99:0/3694408878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d831e80 0x55bd9d833980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 -- 172.21.15.99:0/3694408878 >> 172.21.15.99:0/3694408878 conn(0x55bd9d7fa200 msgr2=0x55bd9d833270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 -- 172.21.15.99:0/3694408878 shutdown_connections 2024-03-23T23:46:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.202+0000 7f05d2237740 1 -- 172.21.15.99:0/3694408878 wait complete. 2024-03-23T23:46:04.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.203+0000 7f05d2237740 1 -- start start 2024-03-23T23:46:04.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.203+0000 7f05d2237740 1 Processor -- start 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d8414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d8379e0 0x55bd9d842400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81abee0 con 0x55bd9d832870 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81ab880 con 0x55bd9d831e80 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a8d80 con 0x55bd9d8379e0 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05d2237740 10 client.0 _ll_register_callbacks cb 0x7f05bc009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47910/0 (socket says 172.21.15.99:47910) 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 -- 172.21.15.99:0/3251542222 learned_addr learned my addr 172.21.15.99:0/3251542222 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d8414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05cd600640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d8379e0 0x55bd9d842400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 -- 172.21.15.99:0/3251542222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d8379e0 msgr2=0x55bd9d842400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d8379e0 0x55bd9d842400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 -- 172.21.15.99:0/3251542222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 msgr2=0x55bd9d8414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d8414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.204+0000 7f05ccc00640 1 -- 172.21.15.99:0/3251542222 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9d8429f0 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ccc00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01e970 tx=0x7f05bc01e690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4afea9fa020b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ae000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc03e070 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f05bc00f8b0 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b40c0a80 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05bc00fab0 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05d2237740 10 client.26012 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bd9d8a3a80 con 0x55bd9d831e80 2024-03-23T23:46:04.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.205+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b40c0a80 con 0x55bd9d831e80 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f05bc041030 con 0x55bd9d831e80 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 10 client.26012 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 20 client.26012 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d8a4370 0x55bd9d8a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 4 client.26012 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:46:04.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd9d79eae0 con 0x55bd9d8a4370 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d8a4370 0x55bd9d8a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05c7e00640 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d8a4370 0x55bd9d8a6790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9069b21bd5faa3 server_cookie=e06e20ecbab24235 in_seq=0 out_seq=0 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.206+0000 7f05ae000640 10 client.26012 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05ae000640 1 -- 172.21.15.99:0/3251542222 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bd9d79eae0 con 0x55bd9d8a4370 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05ae000640 10 client.26012 handle_command_reply: tid=0 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 client.26012 shutdown 2024-03-23T23:46:04.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 msgr2=0x55bd9d840f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01e970 tx=0x7f05bc01e690 comp rx=0 tx=0).stop 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 shutdown_connections 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55bd9d8a4370 0x55bd9d8a6790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd9d8379e0 0x55bd9d842400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd9d832870 0x55bd9d8414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05ae000640 0 client.26012 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 --2- 172.21.15.99:0/3251542222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bd9d831e80 0x55bd9d840f90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 >> 172.21.15.99:0/3251542222 conn(0x55bd9d7fa200 msgr2=0x55bd9d82ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 shutdown_connections 2024-03-23T23:46:04.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 1 -- 172.21.15.99:0/3251542222 wait complete. 2024-03-23T23:46:04.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.208+0000 7f05d2237740 20 client.26012 trim_cache size 0 max 16384 2024-03-23T23:46:04.215 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:04.215 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-23T23:46:04.215 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-23T23:46:04.215 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:04.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 msgr2=0x7f05b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 0x7f05b00a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55bd9d82e740 tx=0x7f05bc014040 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 msgr2=0x7f05c819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05b40349e0 tx=0x7f05b4032450 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 shutdown_connections 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f05b00a2800 0x7f05b00a4cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05c8105490 0x7f05c81a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c8104ad0 0x7f05c819a1b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 --2- 172.21.15.99:0/3065743448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05c81038e0 0x7f05c8199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 >> 172.21.15.99:0/3065743448 conn(0x7f05c80fee50 msgr2=0x7f05c8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 shutdown_connections 2024-03-23T23:46:04.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.209+0000 7f05cf600640 1 -- 172.21.15.99:0/3065743448 wait complete. 2024-03-23T23:46:04.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T23:46:04.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 -- 172.21.15.99:0/1308141505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a0074100 msgr2=0x7fd9a0074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 --2- 172.21.15.99:0/1308141505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a0074100 0x7fd9a0074580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd988009900 tx=0x7fd98802bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 -- 172.21.15.99:0/1308141505 shutdown_connections 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 --2- 172.21.15.99:0/1308141505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a01093d0 0x7fd9a010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 --2- 172.21.15.99:0/1308141505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a0074100 0x7fd9a0074580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.451+0000 7fd9a5600640 1 --2- 172.21.15.99:0/1308141505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 0x7fd9a0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.452+0000 7fd9a5600640 1 -- 172.21.15.99:0/1308141505 >> 172.21.15.99:0/1308141505 conn(0x7fd9a00fca80 msgr2=0x7fd9a00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.452+0000 7fd9a5600640 1 -- 172.21.15.99:0/1308141505 shutdown_connections 2024-03-23T23:46:04.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.452+0000 7fd9a5600640 1 -- 172.21.15.99:0/1308141505 wait complete. 2024-03-23T23:46:04.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.453+0000 7fd9a5600640 1 Processor -- start 2024-03-23T23:46:04.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.453+0000 7fd9a5600640 1 -- start start 2024-03-23T23:46:04.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 0x7fd9a0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a01093d0 0x7fd9a01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a010f3a0 con 0x7fd9a00756e0 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a010f220 con 0x7fd9a01093d0 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a5600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a010f520 con 0x7fd9a0074100 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50578/0 (socket says 172.21.15.99:50578) 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 -- 172.21.15.99:0/3975013766 learned_addr learned my addr 172.21.15.99:0/3975013766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd997400640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 0x7fd9a0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd9a4c00640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a01093d0 0x7fd9a01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 -- 172.21.15.99:0/3975013766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a01093d0 msgr2=0x7fd9a01a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a01093d0 0x7fd9a01a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 -- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 msgr2=0x7fd9a0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 0x7fd9a0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.454+0000 7fd99fe00640 1 -- 172.21.15.99:0/3975013766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a01a1630 con 0x7fd9a0074100 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd99fe00640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd99000ebc0 tx=0x7fd99000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990010430 con 0x7fd9a0074100 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd990010ab0 con 0x7fd9a0074100 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990018b30 con 0x7fd9a0074100 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a01a1920 con 0x7fd9a0074100 2024-03-23T23:46:04.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.455+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a01a1e40 con 0x7fd9a0074100 2024-03-23T23:46:04.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.456+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a0108100 con 0x7fd9a0074100 2024-03-23T23:46:04.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.459+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd990018cd0 con 0x7fd9a0074100 2024-03-23T23:46:04.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.460+0000 7fd99d600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:04.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.460+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd9900cf120 con 0x7fd9a0074100 2024-03-23T23:46:04.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.460+0000 7fd997400640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:04.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.461+0000 7fd997400640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 0x7fd9840a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd98802c250 tx=0x7fd98802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:04.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.464+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd99009c1a0 con 0x7fd9a0074100 2024-03-23T23:46:04.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.778+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9a002d070 con 0x7fd9a0074100 2024-03-23T23:46:04.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.779+0000 7fd99d600640 1 -- 172.21.15.99:0/3975013766 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fd99009baa0 con 0x7fd9a0074100 2024-03-23T23:46:04.780 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:46:04.780 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:46:04.781 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 msgr2=0x7fd9840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 0x7fd9840a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd98802c250 tx=0x7fd98802cf90 comp rx=0 tx=0).stop 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 msgr2=0x7fd9a0199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd99000ebc0 tx=0x7fd99000c2d0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 shutdown_connections 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd9840a2800 0x7fd9840a4cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9a01093d0 0x7fd9a01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9a00756e0 0x7fd9a0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 --2- 172.21.15.99:0/3975013766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9a0074100 0x7fd9a0199a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:04.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 >> 172.21.15.99:0/3975013766 conn(0x7fd9a00fca80 msgr2=0x7fd9a01052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:04.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 shutdown_connections 2024-03-23T23:46:04.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:04.782+0000 7fd9a5600640 1 -- 172.21.15.99:0/3975013766 wait complete. 2024-03-23T23:46:04.810 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:04.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.038+0000 7f7eca000640 1 -- 172.21.15.99:0/3665167914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec41038e0 msgr2=0x7f7ec4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.038+0000 7f7eca000640 1 --2- 172.21.15.99:0/3665167914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec41038e0 0x7f7ec4103d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009900 tx=0x7f7eb402bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 -- 172.21.15.99:0/3665167914 shutdown_connections 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 --2- 172.21.15.99:0/3665167914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4105490 0x7f7ec410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 --2- 172.21.15.99:0/3665167914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec4104ad0 0x7f7ec4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 --2- 172.21.15.99:0/3665167914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec41038e0 0x7f7ec4103d00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 -- 172.21.15.99:0/3665167914 >> 172.21.15.99:0/3665167914 conn(0x7f7ec40fee50 msgr2=0x7f7ec4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 -- 172.21.15.99:0/3665167914 shutdown_connections 2024-03-23T23:46:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.039+0000 7f7eca000640 1 -- 172.21.15.99:0/3665167914 wait complete. 2024-03-23T23:46:05.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.040+0000 7f7eca000640 1 Processor -- start 2024-03-23T23:46:05.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.040+0000 7f7eca000640 1 -- start start 2024-03-23T23:46:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec41038e0 0x7f7ec4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec4105490 0x7f7ec41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec410f580 con 0x7f7ec41038e0 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec410f400 con 0x7f7ec4105490 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7eca000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec410f700 con 0x7f7ec4104ad0 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec41038e0 0x7f7ec4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50592/0 (socket says 172.21.15.99:50592) 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2674066590 learned_addr learned my addr 172.21.15.99:0/2674066590 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec4105490 0x7f7ec41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2674066590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec4105490 msgr2=0x7f7ec41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec4105490 0x7f7ec41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec41038e0 msgr2=0x7f7ec4199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec41038e0 0x7f7ec4199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.041+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2674066590 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec41a1850 con 0x7f7ec4104ad0 2024-03-23T23:46:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d950 tx=0x7f7eb000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7ebbe00640 1 -- 172.21.15.99:0/2674066590 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0017070 con 0x7f7ec4104ad0 2024-03-23T23:46:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec41a1b40 con 0x7f7ec4104ad0 2024-03-23T23:46:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7ebbe00640 1 -- 172.21.15.99:0/2674066590 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0009c30 con 0x7f7ec4104ad0 2024-03-23T23:46:05.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7ebbe00640 1 -- 172.21.15.99:0/2674066590 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000f6b0 con 0x7f7ec4104ad0 2024-03-23T23:46:05.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.042+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41a20b0 con 0x7f7ec4104ad0 2024-03-23T23:46:05.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.045+0000 7f7ebbe00640 1 -- 172.21.15.99:0/2674066590 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7eb00109d0 con 0x7f7ec4104ad0 2024-03-23T23:46:05.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.046+0000 7f7eccc71740 1 Processor -- start 2024-03-23T23:46:05.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.046+0000 7f7eccc71740 1 -- start start 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b64c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7ac12c0 con 0x561fa7b5e3a0 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7a69c90 con 0x561fa7b5c810 2024-03-23T23:46:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7eccc71740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7abd360 con 0x561fa7b5d9e0 2024-03-23T23:46:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47944/0 (socket says 172.21.15.99:47944) 2024-03-23T23:46:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ec3e00640 1 -- 172.21.15.99:0/2003948700 learned_addr learned my addr 172.21.15.99:0/2003948700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b64c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ebbe00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 0x7f7ea80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.047+0000 7f7ec3400640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 -- 172.21.15.99:0/2003948700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 msgr2=0x561fa7b5de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ebbe00640 1 -- 172.21.15.99:0/2674066590 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7eb00ce730 con 0x7f7ec4104ad0 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b5de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 -- 172.21.15.99:0/2003948700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 msgr2=0x561fa7b64c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b64c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 -- 172.21.15.99:0/2003948700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fa7a8e080 con 0x561fa7b5c810 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 0x7f7ea80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3400640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b5de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.048+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc011590 tx=0x7f7ebc011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7830c5a7b4ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.049+0000 7f7eafe00640 1 -- 172.21.15.99:0/2003948700 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc017050 con 0x561fa7b5c810 2024-03-23T23:46:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.049+0000 7f7eafe00640 1 -- 172.21.15.99:0/2003948700 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc00fd20 con 0x561fa7b5c810 2024-03-23T23:46:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.049+0000 7f7eafe00640 1 -- 172.21.15.99:0/2003948700 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00eaa0 con 0x561fa7b5c810 2024-03-23T23:46:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.049+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 0x7f7ea80a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc022000 tx=0x7f7ebc023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 -- 172.21.15.99:0/2003948700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 msgr2=0x561fa7b5cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc011590 tx=0x7f7ebc011a50 comp rx=0 tx=0).stop 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 -- 172.21.15.99:0/2003948700 shutdown_connections 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b5de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2003948700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b5cc30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 -- 172.21.15.99:0/2003948700 >> 172.21.15.99:0/2003948700 conn(0x561fa7b57c00 msgr2=0x561fa7b58000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 -- 172.21.15.99:0/2003948700 shutdown_connections 2024-03-23T23:46:05.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.050+0000 7f7eccc71740 1 -- 172.21.15.99:0/2003948700 wait complete. 2024-03-23T23:46:05.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.051+0000 7f7eccc71740 1 -- start start 2024-03-23T23:46:05.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.051+0000 7f7eccc71740 1 Processor -- start 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5c810 0x561fa7b66aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7b653b0 con 0x561fa7b5c810 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7a7d0d0 con 0x561fa7b5d9e0 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fa7b65530 con 0x561fa7b5e3a0 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7eccc71740 10 client.0 _ll_register_callbacks cb 0x561fa7a81e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47952/0 (socket says 172.21.15.99:47952) 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 -- 172.21.15.99:0/3895877736 learned_addr learned my addr 172.21.15.99:0/3895877736 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 -- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5e3a0 msgr2=0x561fa7b6e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 -- 172.21.15.99:0/3895877736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5c810 msgr2=0x561fa7b66aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5c810 0x561fa7b66aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.052+0000 7f7ec2a00640 1 -- 172.21.15.99:0/3895877736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fa77d53f0 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.053+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009610 tx=0x7f7eb009aa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a659b031fb2fcd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.053+0000 7f7eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.053+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0020040 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.053+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7eb00a3560 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.053+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ebc00fd20 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb009b100 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 10 client.26030 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fa789f7d0 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc00fd20 con 0x561fa7b5d9e0 2024-03-23T23:46:05.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7eb009a550 con 0x561fa7b5d9e0 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 10 client.26030 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 10 client.26030 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 20 client.26030 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.054+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7b725b0 0x561fa7b74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.055+0000 7f7eccc71740 4 client.26030 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.055+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fa7ace800 con 0x561fa7b725b0 2024-03-23T23:46:05.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.055+0000 7f7ec3400640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7b725b0 0x561fa7b74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.055+0000 7f7ec3400640 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7b725b0 0x561fa7b74a20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f647347ba08f5bb server_cookie=95a02cab372b1119 in_seq=0 out_seq=0 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.055+0000 7f7eacc00640 10 client.26030 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eacc00640 1 -- 172.21.15.99:0/3895877736 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561fa7ace800 con 0x561fa7b725b0 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eacc00640 10 client.26030 handle_command_reply: tid=0 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 client.26030 shutdown 2024-03-23T23:46:05.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 msgr2=0x561fa7b66fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009610 tx=0x7f7eb009aa60 comp rx=0 tx=0).stop 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 shutdown_connections 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b66fe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7b725b0 0x561fa7b74a20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 --2- 172.21.15.99:0/3895877736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5c810 0x561fa7b66aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 >> 172.21.15.99:0/3895877736 conn(0x561fa7b57c00 msgr2=0x561fa7b56eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.056+0000 7f7eacc00640 0 client.26030 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.057+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 shutdown_connections 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.057+0000 7f7eccc71740 1 -- 172.21.15.99:0/3895877736 wait complete. 2024-03-23T23:46:05.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.057+0000 7f7eccc71740 20 client.26030 trim_cache size 0 max 16384 2024-03-23T23:46:05.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.071+0000 7f7eccc71740 1 Processor -- start 2024-03-23T23:46:05.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.071+0000 7f7eccc71740 1 -- start start 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41aab80 con 0x561fa7b5e3a0 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41ac500 con 0x561fa7b5c810 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7eccc71740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41ab1e0 con 0x561fa7b5d9e0 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50622/0 (socket says 172.21.15.99:50622) 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2977879888 learned_addr learned my addr 172.21.15.99:0/2977879888 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec3400640 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2977879888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 msgr2=0x561fa7b56d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b56d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2977879888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 msgr2=0x561fa7b792e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b792e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.072+0000 7f7ec2a00640 1 -- 172.21.15.99:0/2977879888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fa7b5d460 con 0x561fa7b5d9e0 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.073+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0012040 tx=0x7f7eb00cc230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54fdc7546e8a1a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.073+0000 7f7eaf400640 1 -- 172.21.15.99:0/2977879888 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0020070 con 0x561fa7b5d9e0 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.073+0000 7f7eaf400640 1 -- 172.21.15.99:0/2977879888 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb00a37f0 con 0x561fa7b5d9e0 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.073+0000 7f7eaf400640 1 -- 172.21.15.99:0/2977879888 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00b9b50 con 0x561fa7b5d9e0 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 -- 172.21.15.99:0/2977879888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 msgr2=0x561fa7b6dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0012040 tx=0x7f7eb00cc230 comp rx=0 tx=0).stop 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 -- 172.21.15.99:0/2977879888 shutdown_connections 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b792e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6dfe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 --2- 172.21.15.99:0/2977879888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5c810 0x561fa7b56d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 -- 172.21.15.99:0/2977879888 >> 172.21.15.99:0/2977879888 conn(0x561fa7b57c00 msgr2=0x561fa7b56620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.074+0000 7f7eccc71740 1 -- 172.21.15.99:0/2977879888 shutdown_connections 2024-03-23T23:46:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.075+0000 7f7eccc71740 1 -- 172.21.15.99:0/2977879888 wait complete. 2024-03-23T23:46:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.075+0000 7f7eccc71740 1 -- start start 2024-03-23T23:46:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.076+0000 7f7eccc71740 1 Processor -- start 2024-03-23T23:46:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50628/0 (socket says 172.21.15.99:50628) 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47972/0 (socket says 172.21.15.99:47972) 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec2a00640 1 -- 172.21.15.99:0/890304367 learned_addr learned my addr 172.21.15.99:0/890304367 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7ec3e00640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41abea0 con 0x561fa7b5e3a0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41ab840 con 0x561fa7b5d9e0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.077+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41a8d40 con 0x561fa7b5c810 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7eccc71740 10 client.0 _ll_register_callbacks cb 0x7f7eb000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 -- 172.21.15.99:0/890304367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 msgr2=0x561fa7b6a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 -- 172.21.15.99:0/890304367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 msgr2=0x561fa7b6d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fa7b6dd30 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7ec3400640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009230 tx=0x7f7eb4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b5df2bd7c9726c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.078+0000 7f7eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004240 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7eb40043e0 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7eb00a37f0 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eb4054050 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eccc71740 10 client.26068 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.079+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fa7bcea00 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb00a37f0 con 0x561fa7b5c810 2024-03-23T23:46:05.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7eb4004a30 con 0x561fa7b5c810 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 10 client.26068 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 10 client.26068 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 20 client.26068 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7bcf400 0x561fa7bd1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 4 client.26068 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fa7ac8b50 con 0x561fa7bcf400 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.080+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7bcf400 0x561fa7bd1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eacc00640 10 client.26068 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7ec2a00640 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7bcf400 0x561fa7bd1820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4afe3cc5c69836d server_cookie=34f34ae64da853be in_seq=0 out_seq=0 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eacc00640 1 -- 172.21.15.99:0/890304367 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fa7ac8b50 con 0x561fa7bcf400 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eacc00640 10 client.26068 handle_command_reply: tid=0 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eccc71740 1 client.26068 shutdown 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 msgr2=0x561fa7b6a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009230 tx=0x7f7eb4000bc0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.081+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 shutdown_connections 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x561fa7bcf400 0x561fa7bd1820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fa7b5e3a0 0x561fa7b6d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561fa7b5d9e0 0x561fa7b6a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 --2- 172.21.15.99:0/890304367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa7b5c810 0x561fa7b6a1e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eacc00640 0 client.26068 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 >> 172.21.15.99:0/890304367 conn(0x561fa7b57c00 msgr2=0x561fa7b56eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 shutdown_connections 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 1 -- 172.21.15.99:0/890304367 wait complete. 2024-03-23T23:46:05.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.082+0000 7f7eccc71740 20 client.26068 trim_cache size 0 max 16384 2024-03-23T23:46:05.087 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:05.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 msgr2=0x7f7ea80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 0x7f7ea80a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc022000 tx=0x7f7ebc023000 comp rx=0 tx=0).stop 2024-03-23T23:46:05.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 msgr2=0x7f7ec419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d950 tx=0x7f7eb000de10 comp rx=0 tx=0).stop 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 shutdown_connections 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f7ea80a2800 0x7f7ea80a4cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec4105490 0x7f7ec41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4104ad0 0x7f7ec419a180 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 --2- 172.21.15.99:0/2674066590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ec41038e0 0x7f7ec4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 >> 172.21.15.99:0/2674066590 conn(0x7f7ec40fee50 msgr2=0x7f7ec4107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 shutdown_connections 2024-03-23T23:46:05.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.083+0000 7f7eca000640 1 -- 172.21.15.99:0/2674066590 wait complete. 2024-03-23T23:46:05.097 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:04 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3975013766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:05.099 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:05.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T23:46:05.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:04 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3975013766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:05.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:04 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3975013766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:05.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-23T23:46:05.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-23T23:46:05.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-03-23T23:46:05.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-23T23:46:05.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-03-23T23:46:05.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-23T23:46:05.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-23T23:46:05.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-03-23T23:46:05.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-23T23:46:05.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-03-23T23:46:05.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-23T23:46:05.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-03-23T23:46:05.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-23T23:46:05.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.296+0000 7f8bcea00640 1 -- 172.21.15.99:0/746947660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc81015a0 msgr2=0x7f8bc81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.296+0000 7f8bcea00640 1 --2- 172.21.15.99:0/746947660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc81015a0 0x7f8bc81019c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009900 tx=0x7f8bbc02bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:05.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 -- 172.21.15.99:0/746947660 shutdown_connections 2024-03-23T23:46:05.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 --2- 172.21.15.99:0/746947660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8103150 0x7f8bc81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 --2- 172.21.15.99:0/746947660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc8102790 0x7f8bc8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 --2- 172.21.15.99:0/746947660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc81015a0 0x7f8bc81019c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 -- 172.21.15.99:0/746947660 >> 172.21.15.99:0/746947660 conn(0x7f8bc80fcb10 msgr2=0x7f8bc80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 -- 172.21.15.99:0/746947660 shutdown_connections 2024-03-23T23:46:05.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.297+0000 7f8bcea00640 1 -- 172.21.15.99:0/746947660 wait complete. 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 Processor -- start 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 -- start start 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 0x7f8bc819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc8103150 0x7f8bc81a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc810d260 con 0x7f8bc8102790 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc810d0e0 con 0x7f8bc81015a0 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcea00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc810d3e0 con 0x7f8bc8103150 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 0x7f8bc819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 0x7f8bc819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37924/0 (socket says 172.21.15.99:37924) 2024-03-23T23:46:05.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:47992/0 (socket says 172.21.15.99:47992) 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bccc00640 1 -- 172.21.15.99:0/3428719048 learned_addr learned my addr 172.21.15.99:0/3428719048 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.298+0000 7f8bce000640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc8103150 0x7f8bc81a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 -- 172.21.15.99:0/3428719048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc8103150 msgr2=0x7f8bc81a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc8103150 0x7f8bc81a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 -- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 msgr2=0x7f8bc819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 0x7f8bc819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 -- 172.21.15.99:0/3428719048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc81a16e0 con 0x7f8bc81015a0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcd600640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009230 tx=0x7f8bbc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bc6000640 1 -- 172.21.15.99:0/3428719048 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc041070 con 0x7f8bc81015a0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.299+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81a1970 con 0x7f8bc81015a0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.300+0000 7f8bc6000640 1 -- 172.21.15.99:0/3428719048 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc0054e0 con 0x7f8bc81015a0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.300+0000 7f8bc6000640 1 -- 172.21.15.99:0/3428719048 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003e40 con 0x7f8bc81015a0 2024-03-23T23:46:05.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.300+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a1e80 con 0x7f8bc81015a0 2024-03-23T23:46:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 Processor -- start 2024-03-23T23:46:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 -- start start 2024-03-23T23:46:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bc6000640 1 -- 172.21.15.99:0/3428719048 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8bbc044070 con 0x7f8bc81015a0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a130e2c0 con 0x5621a13aabc0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a12b6c90 con 0x5621a13ab580 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a130a360 con 0x5621a13a99f0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.303+0000 7f8bce000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50662/0 (socket says 172.21.15.99:50662) 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 -- 172.21.15.99:0/2047747931 learned_addr learned my addr 172.21.15.99:0/2047747931 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bcd600640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 -- 172.21.15.99:0/2047747931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 msgr2=0x5621a13b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 -- 172.21.15.99:0/2047747931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 msgr2=0x5621a13ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 -- 172.21.15.99:0/2047747931 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621a12db080 con 0x5621a13a99f0 2024-03-23T23:46:05.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:05.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.304+0000 7f8bce000640 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009bb0 tx=0x7f8bc000e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce41b1d9bd03196d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.305+0000 7f8bba600640 1 -- 172.21.15.99:0/2047747931 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000ec20 con 0x5621a13a99f0 2024-03-23T23:46:05.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.305+0000 7f8bba600640 1 -- 172.21.15.99:0/2047747931 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0003e50 con 0x5621a13a99f0 2024-03-23T23:46:05.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.305+0000 7f8bba600640 1 -- 172.21.15.99:0/2047747931 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0020620 con 0x5621a13a99f0 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.305+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2047747931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 msgr2=0x5621a13a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.305+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009bb0 tx=0x7f8bc000e3b0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2047747931 shutdown_connections 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2047747931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a9e10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2047747931 >> 172.21.15.99:0/2047747931 conn(0x5621a13a4de0 msgr2=0x5621a13a51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bc6000640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 0x7f8bb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2047747931 shutdown_connections 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2047747931 wait complete. 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bc6000640 1 -- 172.21.15.99:0/3428719048 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8bbc0f04c0 con 0x7f8bc81015a0 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bccc00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 0x7f8bb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.306+0000 7f8bd2dba740 1 -- start start 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 Processor -- start 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bccc00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 0x7f8bb00a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5621a13a5d50 tx=0x7f8bb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13a99f0 0x5621a13b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13aabc0 0x5621a13b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a13b2590 con 0x5621a13a99f0 2024-03-23T23:46:05.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a12ca0d0 con 0x5621a13aabc0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a13b2710 con 0x5621a13ab580 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.307+0000 7f8bd2dba740 10 client.0 _ll_register_callbacks cb 0x5621a12cee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bccc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13aabc0 0x5621a13b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50664/0 (socket says 172.21.15.99:50664) 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 -- 172.21.15.99:0/1286328028 learned_addr learned my addr 172.21.15.99:0/1286328028 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bcd600640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13a99f0 0x5621a13b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 -- 172.21.15.99:0/1286328028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13aabc0 msgr2=0x5621a13b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13aabc0 0x5621a13b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 -- 172.21.15.99:0/1286328028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13a99f0 msgr2=0x5621a13b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13a99f0 0x5621a13b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 -- 172.21.15.99:0/1286328028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621a10223f0 con 0x5621a13ab580 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8bce000640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e8b0 tx=0x7f8bc0004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ad95b579caa34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.308+0000 7f8baea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:05.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003970 con 0x5621a13ab580 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8bc0026070 con 0x5621a13ab580 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bc0003e50 con 0x5621a13ab580 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc0003b10 con 0x5621a13ab580 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8bd2dba740 10 client.26080 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5621a10ec7d0 con 0x5621a13ab580 2024-03-23T23:46:05.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.309+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0003e50 con 0x5621a13ab580 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8bc0005680 con 0x5621a13ab580 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 10 client.26080 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 10 client.26080 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 20 client.26080 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a13bf420 0x5621a13c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 4 client.26080 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621a131b800 con 0x5621a13bf420 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.310+0000 7f8bcd600640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a13bf420 0x5621a13c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.311+0000 7f8baea00640 10 client.26080 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.311+0000 7f8bcd600640 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a13bf420 0x5621a13c1810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b593d5dca32e6eae server_cookie=cbfc5e349b2b7a67 in_seq=0 out_seq=0 2024-03-23T23:46:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8baea00640 1 -- 172.21.15.99:0/1286328028 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5621a131b800 con 0x5621a13bf420 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8baea00640 10 client.26080 handle_command_reply: tid=0 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 client.26080 shutdown 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 msgr2=0x5621a13bb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000e8b0 tx=0x7f8bc0004790 comp rx=0 tx=0).stop 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 shutdown_connections 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a13bf420 0x5621a13c1810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13ab580 0x5621a13bb1c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13aabc0 0x5621a13b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/1286328028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13a99f0 0x5621a13b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 >> 172.21.15.99:0/1286328028 conn(0x5621a13a4de0 msgr2=0x5621a13a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8baea00640 0 client.26080 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.312+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 shutdown_connections 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.313+0000 7f8bd2dba740 1 -- 172.21.15.99:0/1286328028 wait complete. 2024-03-23T23:46:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.313+0000 7f8bd2dba740 20 client.26080 trim_cache size 0 max 16384 2024-03-23T23:46:05.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.327+0000 7f8bd2dba740 1 Processor -- start 2024-03-23T23:46:05.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 -- start start 2024-03-23T23:46:05.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a65a0 con 0x5621a13aabc0 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a7f20 con 0x5621a13ab580 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a6c00 con 0x5621a13a99f0 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bce000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50668/0 (socket says 172.21.15.99:50668) 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bccc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13bb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37956/0 (socket says 172.21.15.99:37956) 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bcd600640 1 -- 172.21.15.99:0/3675121679 learned_addr learned my addr 172.21.15.99:0/3675121679 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.328+0000 7f8bce000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48022/0 (socket says 172.21.15.99:48022) 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 -- 172.21.15.99:0/3675121679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 msgr2=0x5621a13a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 -- 172.21.15.99:0/3675121679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 msgr2=0x5621a13bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 -- 172.21.15.99:0/3675121679 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621a13a5b50 con 0x5621a13ab580 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bce000640 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0017e80 tx=0x7f8bc0017c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d168456d63ae8171 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bb9c00640 1 -- 172.21.15.99:0/3675121679 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0042070 con 0x5621a13ab580 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.329+0000 7f8bb9c00640 1 -- 172.21.15.99:0/3675121679 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bc002f070 con 0x5621a13ab580 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bb9c00640 1 -- 172.21.15.99:0/3675121679 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00044d0 con 0x5621a13ab580 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 -- 172.21.15.99:0/3675121679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 msgr2=0x5621a13a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0017e80 tx=0x7f8bc0017c10 comp rx=0 tx=0).stop 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 -- 172.21.15.99:0/3675121679 shutdown_connections 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13ab580 0x5621a13a38d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13aabc0 0x5621a13bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/3675121679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13a99f0 0x5621a13a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.330+0000 7f8bd2dba740 1 -- 172.21.15.99:0/3675121679 >> 172.21.15.99:0/3675121679 conn(0x5621a13a4de0 msgr2=0x5621a13a8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.331+0000 7f8bd2dba740 1 -- 172.21.15.99:0/3675121679 shutdown_connections 2024-03-23T23:46:05.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.331+0000 7f8bd2dba740 1 -- 172.21.15.99:0/3675121679 wait complete. 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.331+0000 7f8bd2dba740 1 -- start start 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.331+0000 7f8bd2dba740 1 Processor -- start 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 0x5621a13b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 0x5621a13b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a78c0 con 0x5621a13ab580 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a7260 con 0x5621a13a99f0 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a4780 con 0x5621a13aabc0 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bd2dba740 10 client.0 _ll_register_callbacks cb 0x7f8bc0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48030/0 (socket says 172.21.15.99:48030) 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 -- 172.21.15.99:0/2313933444 learned_addr learned my addr 172.21.15.99:0/2313933444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 0x5621a13b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bce000640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 0x5621a13b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 -- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 msgr2=0x5621a13b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 0x5621a13b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 -- 172.21.15.99:0/2313933444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 msgr2=0x5621a13b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 0x5621a13b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bcd600640 1 -- 172.21.15.99:0/2313933444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621a13ba190 con 0x5621a13a99f0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 0x5621a13b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.332+0000 7f8bce000640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 0x5621a13b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8bcd600640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0bcb90 tx=0x7f8bbc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5466fea61a15eb28 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc04b040 con 0x5621a13a99f0 2024-03-23T23:46:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8bbc0bb190 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bc002f070 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bbc0bce90 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8bd2dba740 10 client.26054 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5621a141ac00 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bc002f070 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.333+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8bbc0f00b0 con 0x5621a13a99f0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 10 client.26054 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 10 client.26054 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 20 client.26054 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a141b3c0 0x5621a141d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 4 client.26054 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621a1315b50 con 0x5621a141b3c0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a141b3c0 0x5621a141d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8bccc00640 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a141b3c0 0x5621a141d800 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd4e7ad9618f9ebe server_cookie=fd76fda5a8351a1d in_seq=0 out_seq=0 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.334+0000 7f8baea00640 10 client.26054 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8baea00640 1 -- 172.21.15.99:0/2313933444 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621a1315b50 con 0x5621a141b3c0 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8baea00640 10 client.26054 handle_command_reply: tid=0 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 client.26054 shutdown 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 msgr2=0x5621a13b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0bcb90 tx=0x7f8bbc009260 comp rx=0 tx=0).stop 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 shutdown_connections 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x5621a141b3c0 0x5621a141d800 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621a13ab580 0x5621a13b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621a13aabc0 0x5621a13b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8baea00640 0 client.26054 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 --2- 172.21.15.99:0/2313933444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5621a13a99f0 0x5621a13b8650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 >> 172.21.15.99:0/2313933444 conn(0x5621a13a4de0 msgr2=0x5621a13a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 shutdown_connections 2024-03-23T23:46:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 1 -- 172.21.15.99:0/2313933444 wait complete. 2024-03-23T23:46:05.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.335+0000 7f8bd2dba740 20 client.26054 trim_cache size 0 max 16384 2024-03-23T23:46:05.340 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 msgr2=0x7f8bb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 0x7f8bb00a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5621a13a5d50 tx=0x7f8bb4005f90 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 msgr2=0x7f8bc8199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009230 tx=0x7f8bbc02c660 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 shutdown_connections 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f8bb00a2800 0x7f8bb00a4cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc8103150 0x7f8bc81a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bc8102790 0x7f8bc819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 --2- 172.21.15.99:0/3428719048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bc81015a0 0x7f8bc8199ac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 >> 172.21.15.99:0/3428719048 conn(0x7f8bc80fcb10 msgr2=0x7f8bc80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 shutdown_connections 2024-03-23T23:46:05.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.336+0000 7f8bcea00640 1 -- 172.21.15.99:0/3428719048 wait complete. 2024-03-23T23:46:05.354 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:05.354 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 -- 172.21.15.99:0/2890830757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268102120 msgr2=0x7f4268104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 --2- 172.21.15.99:0/2890830757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268102120 0x7f4268104510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4258009be0 tx=0x7f425802bef0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 -- 172.21.15.99:0/2890830757 shutdown_connections 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 --2- 172.21.15.99:0/2890830757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268107380 0x7f42681097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 --2- 172.21.15.99:0/2890830757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268104a50 0x7f4268106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 --2- 172.21.15.99:0/2890830757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268102120 0x7f4268104510 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 -- 172.21.15.99:0/2890830757 >> 172.21.15.99:0/2890830757 conn(0x7f42680fbb30 msgr2=0x7f42680fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 -- 172.21.15.99:0/2890830757 shutdown_connections 2024-03-23T23:46:05.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.565+0000 7f426de00640 1 -- 172.21.15.99:0/2890830757 wait complete. 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 Processor -- start 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 -- start start 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268102120 0x7f4268199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268107380 0x7f42681a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426810d390 con 0x7f4268102120 2024-03-23T23:46:05.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426810d210 con 0x7f4268104a50 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f426de00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426810d510 con 0x7f4268107380 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.567+0000 7f425e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f4266a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268102120 0x7f4268199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48044/0 (socket says 172.21.15.99:48044) 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 -- 172.21.15.99:0/1363046876 learned_addr learned my addr 172.21.15.99:0/1363046876 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f4267400640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268107380 0x7f42681a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 -- 172.21.15.99:0/1363046876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268107380 msgr2=0x7f42681a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268107380 0x7f42681a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 -- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268102120 msgr2=0x7f4268199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268102120 0x7f4268199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 -- 172.21.15.99:0/1363046876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42681a1850 con 0x7f4268104a50 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.568+0000 7f425e000640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.569+0000 7f426ca00640 1 -- 172.21.15.99:0/1363046876 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000ccc0 con 0x7f4268104a50 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.569+0000 7f426ca00640 1 -- 172.21.15.99:0/1363046876 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4250005500 con 0x7f4268104a50 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.569+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42681a1b40 con 0x7f4268104a50 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.569+0000 7f426ca00640 1 -- 172.21.15.99:0/1363046876 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250010690 con 0x7f4268104a50 2024-03-23T23:46:05.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.569+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42681a2090 con 0x7f4268104a50 2024-03-23T23:46:05.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.572+0000 7f426ca00640 1 -- 172.21.15.99:0/1363046876 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f42500046c0 con 0x7f4268104a50 2024-03-23T23:46:05.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.573+0000 7f4270ba6740 1 Processor -- start 2024-03-23T23:46:05.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.573+0000 7f4270ba6740 1 -- start start 2024-03-23T23:46:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b14e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b1bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4a782c0 con 0x5588a4b149e0 2024-03-23T23:46:05.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4a20c90 con 0x5588a4b13810 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4270ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4a74360 con 0x5588a4b153a0 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48050/0 (socket says 172.21.15.99:48050) 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 -- 172.21.15.99:0/2223704278 learned_addr learned my addr 172.21.15.99:0/2223704278 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4266a00640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f425e000640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b1bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 -- 172.21.15.99:0/2223704278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 msgr2=0x5588a4b1bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b1bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 -- 172.21.15.99:0/2223704278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 msgr2=0x5588a4b14e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b14e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4267400640 1 -- 172.21.15.99:0/2223704278 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a4a45080 con 0x5588a4b13810 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f425e000640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b1bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.574+0000 7f4266a00640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b14e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:05.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f426ca00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 0x7f424c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f4267400640 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4260009bc0 tx=0x7f42600117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9307d4278209b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f426ca00640 1 -- 172.21.15.99:0/1363046876 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f42500ceea0 con 0x7f4268104a50 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f4266a00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 0x7f424c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f4257e00640 1 -- 172.21.15.99:0/2223704278 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260012450 con 0x5588a4b13810 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f4257e00640 1 -- 172.21.15.99:0/2223704278 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4260012ad0 con 0x5588a4b13810 2024-03-23T23:46:05.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.575+0000 7f4257e00640 1 -- 172.21.15.99:0/2223704278 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000fbb0 con 0x5588a4b13810 2024-03-23T23:46:05.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- 172.21.15.99:0/2223704278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 msgr2=0x5588a4b13c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4260009bc0 tx=0x7f42600117b0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4266a00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 0x7f424c0a4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f426819b110 tx=0x7f4258000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- 172.21.15.99:0/2223704278 shutdown_connections 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b1bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b14e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2223704278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b13c30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- 172.21.15.99:0/2223704278 >> 172.21.15.99:0/2223704278 conn(0x5588a4b0ec00 msgr2=0x5588a4b0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- 172.21.15.99:0/2223704278 shutdown_connections 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- 172.21.15.99:0/2223704278 wait complete. 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.576+0000 7f4270ba6740 1 -- start start 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 Processor -- start 2024-03-23T23:46:05.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b13810 0x5588a4b1dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 0x5588a4b25060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4b1c3b0 con 0x5588a4b13810 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4a340d0 con 0x5588a4b153a0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a4b1c530 con 0x5588a4b149e0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4270ba6740 10 client.0 _ll_register_callbacks cb 0x5588a4a38e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f425e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4267400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 0x5588a4b25060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f425e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50720/0 (socket says 172.21.15.99:50720) 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f425e000640 1 -- 172.21.15.99:0/1860127202 learned_addr learned my addr 172.21.15.99:0/1860127202 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.577+0000 7f4266a00640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b13810 0x5588a4b1dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 -- 172.21.15.99:0/1860127202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 msgr2=0x5588a4b25060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 0x5588a4b25060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 -- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b13810 msgr2=0x5588a4b1dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b13810 0x5588a4b1dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 -- 172.21.15.99:0/1860127202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588a478c3f0 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f4267400640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 0x5588a4b25060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f425e000640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42500a3a90 tx=0x7f4250097430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc450033bb107fef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f4254c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.578+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425009b5f0 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4260012ad0 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f42500a3d30 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4250099960 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 10 client.26095 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588a48567d0 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4260012ad0 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f425009cc70 con 0x5588a4b149e0 2024-03-23T23:46:05.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 10 client.26095 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 10 client.26095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 20 client.26095 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b29530 0x5588a4b2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 4 client.26095 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.579+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588a4a85800 con 0x5588a4b29530 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.580+0000 7f4266a00640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b29530 0x5588a4b2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.580+0000 7f4266a00640 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b29530 0x5588a4b2b950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=431413d3a9f4410a server_cookie=927699040697a8ef in_seq=0 out_seq=0 2024-03-23T23:46:05.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.580+0000 7f4254c00640 10 client.26095 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:05.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4254c00640 1 -- 172.21.15.99:0/1860127202 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588a4a85800 con 0x5588a4b29530 2024-03-23T23:46:05.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4254c00640 10 client.26095 handle_command_reply: tid=0 2024-03-23T23:46:05.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4270ba6740 1 client.26095 shutdown 2024-03-23T23:46:05.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 msgr2=0x5588a4b1e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42500a3a90 tx=0x7f4250097430 comp rx=0 tx=0).stop 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 shutdown_connections 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.581+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b29530 0x5588a4b2b950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b153a0 0x5588a4b25060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b1e020 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 --2- 172.21.15.99:0/1860127202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b13810 0x5588a4b1dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 >> 172.21.15.99:0/1860127202 conn(0x5588a4b0ec00 msgr2=0x5588a4b0deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4254c00640 0 client.26095 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 shutdown_connections 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 1 -- 172.21.15.99:0/1860127202 wait complete. 2024-03-23T23:46:05.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.582+0000 7f4270ba6740 20 client.26095 trim_cache size 0 max 16384 2024-03-23T23:46:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.599+0000 7f4270ba6740 1 Processor -- start 2024-03-23T23:46:05.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.599+0000 7f4270ba6740 1 -- start start 2024-03-23T23:46:05.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b153a0 0x5588a4b17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681aace0 con 0x5588a4b153a0 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681ac660 con 0x5588a4b13810 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4270ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681ab340 con 0x5588a4b149e0 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48068/0 (socket says 172.21.15.99:48068) 2024-03-23T23:46:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4266a00640 1 -- 172.21.15.99:0/2748231493 learned_addr learned my addr 172.21.15.99:0/2748231493 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4267400640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b153a0 0x5588a4b17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 -- 172.21.15.99:0/2748231493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 msgr2=0x5588a4b17a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 -- 172.21.15.99:0/2748231493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b153a0 msgr2=0x5588a4b17fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b153a0 0x5588a4b17fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f425e000640 1 -- 172.21.15.99:0/2748231493 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a4b0a4e0 con 0x5588a4b149e0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.600+0000 7f4266a00640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.601+0000 7f425e000640 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b25020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f425009b7b0 tx=0x7f4250009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aaebe63a1f091ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.601+0000 7f4257400640 1 -- 172.21.15.99:0/2748231493 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425009b180 con 0x5588a4b149e0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.601+0000 7f4257400640 1 -- 172.21.15.99:0/2748231493 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f425009f970 con 0x5588a4b149e0 2024-03-23T23:46:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.601+0000 7f4257400640 1 -- 172.21.15.99:0/2748231493 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425009b360 con 0x5588a4b149e0 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 -- 172.21.15.99:0/2748231493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 msgr2=0x5588a4b25020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b25020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f425009b7b0 tx=0x7f4250009740 comp rx=0 tx=0).stop 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 -- 172.21.15.99:0/2748231493 shutdown_connections 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b153a0 0x5588a4b17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b149e0 0x5588a4b25020 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2748231493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b17a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 -- 172.21.15.99:0/2748231493 >> 172.21.15.99:0/2748231493 conn(0x5588a4b0ec00 msgr2=0x5588a4b126f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 -- 172.21.15.99:0/2748231493 shutdown_connections 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.602+0000 7f4270ba6740 1 -- 172.21.15.99:0/2748231493 wait complete. 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 -- start start 2024-03-23T23:46:05.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 Processor -- start 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681ac000 con 0x5588a4b149e0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681ab9a0 con 0x5588a4b13810 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.603+0000 7f4270ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42681a8ea0 con 0x5588a4b153a0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4270ba6740 10 client.0 _ll_register_callbacks cb 0x7f425000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48082/0 (socket says 172.21.15.99:48082) 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 -- 172.21.15.99:0/2531344177 learned_addr learned my addr 172.21.15.99:0/2531344177 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f425e000640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4267400640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 -- 172.21.15.99:0/2531344177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 msgr2=0x5588a4b24070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b24070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 -- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 msgr2=0x5588a4b210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 -- 172.21.15.99:0/2531344177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588a4b24660 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4266a00640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4258034fd0 tx=0x7f425802bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f81ed18efe2890fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.604+0000 7f4254c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258053d70 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f425803a040 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f425009f970 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4258063050 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4270ba6740 10 client.26072 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588a4b85140 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.605+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f425009f970 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4258004040 con 0x5588a4b13810 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 10 client.26072 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 10 client.26072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:46:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 20 client.26072 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b85a70 0x5588a4b87e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 4 client.26072 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:05.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588a4a7fb50 con 0x5588a4b85a70 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.606+0000 7f425e000640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b85a70 0x5588a4b87e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f425e000640 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b85a70 0x5588a4b87e90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e40bc1d16f97b577 server_cookie=5bedf82d58caac4 in_seq=0 out_seq=0 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4254c00640 10 client.26072 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4254c00640 1 -- 172.21.15.99:0/2531344177 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588a4a7fb50 con 0x5588a4b85a70 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4254c00640 10 client.26072 handle_command_reply: tid=0 2024-03-23T23:46:05.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 client.26072 shutdown 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 msgr2=0x5588a4b20b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4258034fd0 tx=0x7f425802bef0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 shutdown_connections 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x5588a4b85a70 0x5588a4b87e90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588a4b153a0 0x5588a4b24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5588a4b149e0 0x5588a4b210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4254c00640 0 client.26072 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 --2- 172.21.15.99:0/2531344177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5588a4b13810 0x5588a4b20b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.607+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 >> 172.21.15.99:0/2531344177 conn(0x5588a4b0ec00 msgr2=0x5588a4b0d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 shutdown_connections 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f4270ba6740 1 -- 172.21.15.99:0/2531344177 wait complete. 2024-03-23T23:46:05.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f4270ba6740 20 client.26072 trim_cache size 0 max 16384 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 msgr2=0x7f424c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 0x7f424c0a4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f426819b110 tx=0x7f4258000f40 comp rx=0 tx=0).stop 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 msgr2=0x7f426819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 shutdown_connections 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f424c0a2880 0x7f424c0a4d40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4268107380 0x7f42681a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.608+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4268104a50 0x7f426819a180 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.609+0000 7f426de00640 1 --2- 172.21.15.99:0/1363046876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4268102120 0x7f4268199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.609+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 >> 172.21.15.99:0/1363046876 conn(0x7f42680fbb30 msgr2=0x7f4268101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.609+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 shutdown_connections 2024-03-23T23:46:05.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.609+0000 7f426de00640 1 -- 172.21.15.99:0/1363046876 wait complete. 2024-03-23T23:46:05.632 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:05.632 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T23:46:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 -- 172.21.15.99:0/222918349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec1016b0 msgr2=0x7fe0ec101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 --2- 172.21.15.99:0/222918349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec101ad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009900 tx=0x7fe0e002bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 -- 172.21.15.99:0/222918349 shutdown_connections 2024-03-23T23:46:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 --2- 172.21.15.99:0/222918349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec103260 0x7fe0ec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 --2- 172.21.15.99:0/222918349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 --2- 172.21.15.99:0/222918349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec101ad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 -- 172.21.15.99:0/222918349 >> 172.21.15.99:0/222918349 conn(0x7fe0ec0fcc40 msgr2=0x7fe0ec0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 -- 172.21.15.99:0/222918349 shutdown_connections 2024-03-23T23:46:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.826+0000 7fe0f4400640 1 -- 172.21.15.99:0/222918349 wait complete. 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.827+0000 7fe0f4400640 1 Processor -- start 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 -- start start 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec103260 0x7fe0ec103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec10d350 con 0x7fe0ec103260 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec10d1d0 con 0x7fe0ec1028a0 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f4400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec10d4d0 con 0x7fe0ec1016b0 2024-03-23T23:46:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48102/0 (socket says 172.21.15.99:48102) 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f1000640 1 -- 172.21.15.99:0/780130652 learned_addr learned my addr 172.21.15.99:0/780130652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.828+0000 7fe0f2400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec103260 0x7fe0ec103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 -- 172.21.15.99:0/780130652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec1016b0 msgr2=0x7fe0ec107430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec107430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 -- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec103260 msgr2=0x7fe0ec103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec103260 0x7fe0ec103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 -- 172.21.15.99:0/780130652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec104860 con 0x7fe0ec1028a0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f1000640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9f0 tx=0x7fe0d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0ea000640 1 -- 172.21.15.99:0/780130652 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800cd80 con 0x7fe0ec1028a0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0ea000640 1 -- 172.21.15.99:0/780130652 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0d8010910 con 0x7fe0ec1028a0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0ea000640 1 -- 172.21.15.99:0/780130652 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8018990 con 0x7fe0ec1028a0 2024-03-23T23:46:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.829+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec1a6060 con 0x7fe0ec1028a0 2024-03-23T23:46:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.830+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec1a6470 con 0x7fe0ec1028a0 2024-03-23T23:46:05.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.833+0000 7fe0f7186740 1 Processor -- start 2024-03-23T23:46:05.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 -- start start 2024-03-23T23:46:05.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f172e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1733a0 0x55eb6f179c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f0d62c0 con 0x55eb6f1729e0 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f07ec90 con 0x55eb6f171810 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.834+0000 7fe0f7186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f0d2360 con 0x55eb6f1733a0 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48114/0 (socket says 172.21.15.99:48114) 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 -- 172.21.15.99:0/2677298923 learned_addr learned my addr 172.21.15.99:0/2677298923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f172e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0ea000640 1 -- 172.21.15.99:0/780130652 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe0d8010ac0 con 0x7fe0ec1028a0 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 -- 172.21.15.99:0/2677298923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1733a0 msgr2=0x55eb6f179c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1733a0 0x55eb6f179c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 -- 172.21.15.99:0/2677298923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 msgr2=0x55eb6f172e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f172e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f2400640 1 -- 172.21.15.99:0/2677298923 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb6f0a3080 con 0x55eb6f171810 2024-03-23T23:46:05.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.835+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f172e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.836+0000 7fe0f2400640 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4010370 tx=0x7fe0e4010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ac017ee07d21f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.836+0000 7fe0dea00640 1 -- 172.21.15.99:0/2677298923 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4016070 con 0x55eb6f171810 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.836+0000 7fe0dea00640 1 -- 172.21.15.99:0/2677298923 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4004d20 con 0x55eb6f171810 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.836+0000 7fe0dea00640 1 -- 172.21.15.99:0/2677298923 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4005480 con 0x55eb6f171810 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.837+0000 7fe0ea000640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 0x7fe0d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.837+0000 7fe0f7186740 1 -- 172.21.15.99:0/2677298923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 msgr2=0x55eb6f171c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.837+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4010370 tx=0x7fe0e4010830 comp rx=0 tx=0).stop 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.837+0000 7fe0ea000640 1 -- 172.21.15.99:0/780130652 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0d80ceee0 con 0x7fe0ec1028a0 2024-03-23T23:46:05.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 0x7fe0d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 -- 172.21.15.99:0/2677298923 shutdown_connections 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1733a0 0x55eb6f179c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f172e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2677298923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f171c30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 -- 172.21.15.99:0/2677298923 >> 172.21.15.99:0/2677298923 conn(0x55eb6f16cc00 msgr2=0x55eb6f16d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 -- 172.21.15.99:0/2677298923 shutdown_connections 2024-03-23T23:46:05.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.838+0000 7fe0f7186740 1 -- 172.21.15.99:0/2677298923 wait complete. 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 -- start start 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 0x7fe0d40a4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe0e002c170 tx=0x7fe0e002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 Processor -- start 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f18cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f18d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f194050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f18b3c0 con 0x55eb6f1733a0 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f0920d0 con 0x55eb6f171810 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb6f18b540 con 0x55eb6f1729e0 2024-03-23T23:46:05.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.839+0000 7fe0f7186740 10 client.0 _ll_register_callbacks cb 0x55eb6f096e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f18cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f18d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f18cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48118/0 (socket says 172.21.15.99:48118) 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1a00640 1 -- 172.21.15.99:0/30433961 learned_addr learned my addr 172.21.15.99:0/30433961 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f2400640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f194050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 -- 172.21.15.99:0/30433961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 msgr2=0x55eb6f18cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f18cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 -- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 msgr2=0x55eb6f194050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f194050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 -- 172.21.15.99:0/30433961 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb6edea3f0 con 0x55eb6f1729e0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0f1000640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f18d010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9c0 tx=0x7fe0d809bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3762425f4a93df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.840+0000 7fe0d3400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80a3fa0 con 0x55eb6f1729e0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0d80afbe0 con 0x55eb6f1729e0 2024-03-23T23:46:05.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0e4004d20 con 0x55eb6f1729e0 2024-03-23T23:46:05.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d80afde0 con 0x55eb6f1729e0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0f7186740 10 client.26110 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.841+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eb6eeb47d0 con 0x55eb6f1729e0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4004d20 con 0x55eb6f1729e0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe0d8018b30 con 0x55eb6f1729e0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 10 client.26110 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 10 client.26110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 20 client.26110 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f198560 0x55eb6f19a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 4 client.26110 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb6f0e3800 con 0x55eb6f198560 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.842+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f198560 0x55eb6f19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.843+0000 7fe0d3400640 10 client.26110 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.843+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f198560 0x55eb6f19a980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3da3d816a9275bb server_cookie=7d5e565b8378987 in_seq=0 out_seq=0 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0d3400640 1 -- 172.21.15.99:0/30433961 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eb6f0e3800 con 0x55eb6f198560 2024-03-23T23:46:05.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0d3400640 10 client.26110 handle_command_reply: tid=0 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 client.26110 shutdown 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 msgr2=0x55eb6f18d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f18d010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9c0 tx=0x7fe0d809bf00 comp rx=0 tx=0).stop 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 shutdown_connections 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f198560 0x55eb6f19a980 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f194050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0d3400640 0 client.26110 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f18d010 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 --2- 172.21.15.99:0/30433961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f18cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 >> 172.21.15.99:0/30433961 conn(0x55eb6f16cc00 msgr2=0x55eb6f16beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 shutdown_connections 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.844+0000 7fe0f7186740 1 -- 172.21.15.99:0/30433961 wait complete. 2024-03-23T23:46:05.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.845+0000 7fe0f7186740 20 client.26110 trim_cache size 0 max 16384 2024-03-23T23:46:05.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.863+0000 7fe0f7186740 1 Processor -- start 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.863+0000 7fe0f7186740 1 -- start start 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.863+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f16b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f16bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f7186740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1a6970 con 0x55eb6f1733a0 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f7186740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1a82f0 con 0x55eb6f171810 2024-03-23T23:46:05.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f7186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1a6fd0 con 0x55eb6f1729e0 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f2400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f16bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f16b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50796/0 (socket says 172.21.15.99:50796) 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 -- 172.21.15.99:0/705666501 learned_addr learned my addr 172.21.15.99:0/705666501 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 -- 172.21.15.99:0/705666501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 msgr2=0x55eb6f16b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f16b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 -- 172.21.15.99:0/705666501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 msgr2=0x55eb6f16bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f16bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1000640 1 -- 172.21.15.99:0/705666501 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb6f179590 con 0x55eb6f1729e0 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.864+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f16b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.865+0000 7fe0f1000640 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe0d809b500 tx=0x7fe0d8099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8257d87ba12e1771 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.865+0000 7fe0de000640 1 -- 172.21.15.99:0/705666501 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8086110 con 0x55eb6f1729e0 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.865+0000 7fe0de000640 1 -- 172.21.15.99:0/705666501 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0d809cb30 con 0x55eb6f1729e0 2024-03-23T23:46:05.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.865+0000 7fe0de000640 1 -- 172.21.15.99:0/705666501 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80a3b10 con 0x55eb6f1729e0 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- 172.21.15.99:0/705666501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 msgr2=0x55eb6f194010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe0d809b500 tx=0x7fe0d8099d10 comp rx=0 tx=0).stop 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- 172.21.15.99:0/705666501 shutdown_connections 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1733a0 0x55eb6f16bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f1729e0 0x55eb6f194010 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 --2- 172.21.15.99:0/705666501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f171810 0x55eb6f16b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- 172.21.15.99:0/705666501 >> 172.21.15.99:0/705666501 conn(0x55eb6f16cc00 msgr2=0x55eb6f18cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- 172.21.15.99:0/705666501 shutdown_connections 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- 172.21.15.99:0/705666501 wait complete. 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.866+0000 7fe0f7186740 1 -- start start 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 Processor -- start 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f18fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f1733a0 0x55eb6f192e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1a7c90 con 0x55eb6f1729e0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1a7630 con 0x55eb6f1733a0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec071ca0 con 0x55eb6f171810 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f7186740 10 client.0 _ll_register_callbacks cb 0x7fe0d8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50802/0 (socket says 172.21.15.99:50802) 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f1a00640 1 -- 172.21.15.99:0/2874296078 learned_addr learned my addr 172.21.15.99:0/2874296078 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.867+0000 7fe0f1000640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f18fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f2400640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f1733a0 0x55eb6f192e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 -- 172.21.15.99:0/2874296078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f1733a0 msgr2=0x55eb6f192e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f1733a0 0x55eb6f192e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 -- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 msgr2=0x55eb6f18fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f18fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 -- 172.21.15.99:0/2874296078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb6f193430 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f1a00640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe0e002c7b0 tx=0x7fe0e0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8fe6b8d8a984088 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0d3400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00046c0 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0e0004860 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.868+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0d809cb30 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e0065050 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 10 client.26116 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eb6f1f4060 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0d809cb30 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe0e00569c0 con 0x55eb6f171810 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 10 client.26116 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 10 client.26116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:46:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 20 client.26116 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f1f4a10 0x55eb6f1f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 4 client.26116 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.869+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb6f0ddb50 con 0x55eb6f1f4a10 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0f1000640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f1f4a10 0x55eb6f1f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0f1000640 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f1f4a10 0x55eb6f1f6e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6817b74039dda85 server_cookie=a64d14c283ea3f7f in_seq=0 out_seq=0 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0d3400640 10 client.26116 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0d3400640 1 -- 172.21.15.99:0/2874296078 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb6f0ddb50 con 0x55eb6f1f4a10 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0d3400640 10 client.26116 handle_command_reply: tid=0 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.870+0000 7fe0f7186740 1 client.26116 shutdown 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 msgr2=0x55eb6f18f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe0e002c7b0 tx=0x7fe0e0009260 comp rx=0 tx=0).stop 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 shutdown_connections 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55eb6f1f4a10 0x55eb6f1f6e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb6f1733a0 0x55eb6f192e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eb6f1729e0 0x55eb6f18fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 --2- 172.21.15.99:0/2874296078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eb6f171810 0x55eb6f18f930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0d3400640 0 client.26116 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 >> 172.21.15.99:0/2874296078 conn(0x55eb6f16cc00 msgr2=0x55eb6f16fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 shutdown_connections 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 1 -- 172.21.15.99:0/2874296078 wait complete. 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.871+0000 7fe0f7186740 20 client.26116 trim_cache size 0 max 16384 2024-03-23T23:46:05.874 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 msgr2=0x7fe0d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 0x7fe0d40a4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe0e002c170 tx=0x7fe0e002df90 comp rx=0 tx=0).stop 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 msgr2=0x7fe0ec107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9f0 tx=0x7fe0d800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 shutdown_connections 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe0d40a2880 0x7fe0d40a4d40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0ec103260 0x7fe0ec103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0ec1028a0 0x7fe0ec107970 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 --2- 172.21.15.99:0/780130652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0ec1016b0 0x7fe0ec107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 >> 172.21.15.99:0/780130652 conn(0x7fe0ec0fcc40 msgr2=0x7fe0ec0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 shutdown_connections 2024-03-23T23:46:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:05.872+0000 7fe0f4400640 1 -- 172.21.15.99:0/780130652 wait complete. 2024-03-23T23:46:05.887 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:05 smithi099 ceph-mon[31715]: pgmap v466: 97 pgs: 97 active+clean; 148 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 837 KiB/s wr, 339 op/s 2024-03-23T23:46:05.889 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:05.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T23:46:06.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 -- 172.21.15.99:0/1783735736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c100700 msgr2=0x7f5e0c100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 --2- 172.21.15.99:0/1783735736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c100700 0x7f5e0c100b80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009900 tx=0x7f5df402bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 -- 172.21.15.99:0/1783735736 shutdown_connections 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 --2- 172.21.15.99:0/1783735736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c1010c0 0x7f5e0c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 --2- 172.21.15.99:0/1783735736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c100700 0x7f5e0c100b80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 --2- 172.21.15.99:0/1783735736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 -- 172.21.15.99:0/1783735736 >> 172.21.15.99:0/1783735736 conn(0x7f5e0c0fb2d0 msgr2=0x7f5e0c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 -- 172.21.15.99:0/1783735736 shutdown_connections 2024-03-23T23:46:06.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.075+0000 7f5e10e00640 1 -- 172.21.15.99:0/1783735736 wait complete. 2024-03-23T23:46:06.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.076+0000 7f5e10e00640 1 Processor -- start 2024-03-23T23:46:06.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.076+0000 7f5e10e00640 1 -- start start 2024-03-23T23:46:06.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.076+0000 7f5e10e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e10e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c100700 0x7f5e0c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e10e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1010c0 0x7f5e0c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e10e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c10f5b0 con 0x7f5e0c100700 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e10e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c10f430 con 0x7f5e0c1010c0 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e10e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c10f730 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50832/0 (socket says 172.21.15.99:50832) 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 -- 172.21.15.99:0/524541704 learned_addr learned my addr 172.21.15.99:0/524541704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09400640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c100700 0x7f5e0c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e0a800640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1010c0 0x7f5e0c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 -- 172.21.15.99:0/524541704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1010c0 msgr2=0x7f5e0c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1010c0 0x7f5e0c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 -- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c100700 msgr2=0x7f5e0c19a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c100700 0x7f5e0c19a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.077+0000 7f5e09e00640 1 -- 172.21.15.99:0/524541704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c1a1880 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e09e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00ebc0 tx=0x7f5dfc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e02a00640 1 -- 172.21.15.99:0/524541704 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc010430 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e02a00640 1 -- 172.21.15.99:0/524541704 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc010ab0 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e02a00640 1 -- 172.21.15.99:0/524541704 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc018b30 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c1a1b70 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.078+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c1a2110 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.081+0000 7f5e02a00640 1 -- 172.21.15.99:0/524541704 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5dfc018cd0 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.082+0000 7f5e13bd4740 1 Processor -- start 2024-03-23T23:46:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.082+0000 7f5e13bd4740 1 -- start start 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.082+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 0x558221d38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.082+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d399e0 0x558221d39e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221c9d2c0 con 0x558221d399e0 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221c45c90 con 0x558221d3a3a0 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221c99360 con 0x558221d38810 2024-03-23T23:46:06.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e0a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 0x558221d38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e0a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 0x558221d38c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50844/0 (socket says 172.21.15.99:50844) 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e0a800640 1 -- 172.21.15.99:0/138576362 learned_addr learned my addr 172.21.15.99:0/138576362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09e00640 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d399e0 0x558221d39e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e02a00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 0x7f5df00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 -- 172.21.15.99:0/138576362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 msgr2=0x558221d38c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 0x558221d38c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 -- 172.21.15.99:0/138576362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d399e0 msgr2=0x558221d39e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d399e0 0x558221d39e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e09400640 1 -- 172.21.15.99:0/138576362 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558221c6a080 con 0x558221d3a3a0 2024-03-23T23:46:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e02a00640 1 -- 172.21.15.99:0/524541704 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0cf190 con 0x7f5e0c0ff9a0 2024-03-23T23:46:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.083+0000 7f5e0a800640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 0x7f5df00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.084+0000 7f5e09400640 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d40c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009230 tx=0x7f5df402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f698bff575d169 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.084+0000 7f5dfb400640 1 -- 172.21.15.99:0/138576362 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df402c980 con 0x558221d3a3a0 2024-03-23T23:46:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.084+0000 7f5dfb400640 1 -- 172.21.15.99:0/138576362 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df402cb20 con 0x558221d3a3a0 2024-03-23T23:46:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.084+0000 7f5dfb400640 1 -- 172.21.15.99:0/138576362 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4004920 con 0x558221d3a3a0 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.084+0000 7f5e0a800640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 0x7f5df00a4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x558221d398e0 tx=0x7f5e040046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 -- 172.21.15.99:0/138576362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 msgr2=0x558221d40c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d40c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009230 tx=0x7f5df402c670 comp rx=0 tx=0).stop 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 -- 172.21.15.99:0/138576362 shutdown_connections 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d40c30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d399e0 0x558221d39e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/138576362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d38810 0x558221d38c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 -- 172.21.15.99:0/138576362 >> 172.21.15.99:0/138576362 conn(0x558221d33c00 msgr2=0x558221d34000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.085+0000 7f5e13bd4740 1 -- 172.21.15.99:0/138576362 shutdown_connections 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.086+0000 7f5e13bd4740 1 -- 172.21.15.99:0/138576362 wait complete. 2024-03-23T23:46:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.086+0000 7f5e13bd4740 1 -- start start 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.086+0000 7f5e13bd4740 1 Processor -- start 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d3a3a0 0x558221d49f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221d41350 con 0x558221d3a3a0 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221c590d0 con 0x558221d38810 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558221d414d0 con 0x558221d399e0 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e13bd4740 10 client.0 _ll_register_callbacks cb 0x558221c5de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48170/0 (socket says 172.21.15.99:48170) 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e09e00640 1 -- 172.21.15.99:0/2632162839 learned_addr learned my addr 172.21.15.99:0/2632162839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.087+0000 7f5e09400640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e0a800640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d3a3a0 0x558221d49f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 -- 172.21.15.99:0/2632162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 msgr2=0x558221d42f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d42f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 -- 172.21.15.99:0/2632162839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d3a3a0 msgr2=0x558221d49f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d3a3a0 0x558221d49f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 -- 172.21.15.99:0/2632162839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582219b13f0 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09400640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5e09e00640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ccc10 tx=0x7f5dfc09b5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487e3238882ebeca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5defe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.088+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0a3fc0 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0d1110 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df402cb20 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc09c1b0 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 10 client.26102 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558221a7b7d0 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df402cb20 con 0x558221d38810 2024-03-23T23:46:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5dfc09af00 con 0x558221d38810 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 10 client.26102 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 20 client.26102 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221d4e380 0x558221d507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 4 client.26102 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.089+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558221caa800 con 0x558221d4e380 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.090+0000 7f5e09400640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221d4e380 0x558221d507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.090+0000 7f5defe00640 10 client.26102 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:06.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.090+0000 7f5e09400640 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221d4e380 0x558221d507f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c6672672264275 server_cookie=320f2a5abb0f85f4 in_seq=0 out_seq=0 2024-03-23T23:46:06.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.091+0000 7f5defe00640 1 -- 172.21.15.99:0/2632162839 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558221caa800 con 0x558221d4e380 2024-03-23T23:46:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.091+0000 7f5defe00640 10 client.26102 handle_command_reply: tid=0 2024-03-23T23:46:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.091+0000 7f5e13bd4740 1 client.26102 shutdown 2024-03-23T23:46:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 msgr2=0x558221d429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ccc10 tx=0x7f5dfc09b5f0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 shutdown_connections 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221d4e380 0x558221d507f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d3a3a0 0x558221d49f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2632162839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d38810 0x558221d429c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 >> 172.21.15.99:0/2632162839 conn(0x558221d33c00 msgr2=0x558221d32eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5defe00640 0 client.26102 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 shutdown_connections 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2632162839 wait complete. 2024-03-23T23:46:06.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.092+0000 7f5e13bd4740 20 client.26102 trim_cache size 0 max 16384 2024-03-23T23:46:06.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 Processor -- start 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 -- start start 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d3cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d3d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1aabd0 con 0x558221d38810 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1ac550 con 0x558221d3a3a0 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1ab230 con 0x558221d399e0 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e09400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d3cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e09400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50862/0 (socket says 172.21.15.99:50862) 2024-03-23T23:46:06.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.118+0000 7f5e09400640 1 -- 172.21.15.99:0/2072589101 learned_addr learned my addr 172.21.15.99:0/2072589101 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e0a800640 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d3d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 -- 172.21.15.99:0/2072589101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 msgr2=0x558221d3d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d3d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 -- 172.21.15.99:0/2072589101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 msgr2=0x558221d3cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d3cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 -- 172.21.15.99:0/2072589101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558221d41290 con 0x558221d399e0 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5e09400640 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x558221d34b70 tx=0x7f5df4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e433c6f475ca5c64 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5dfaa00640 1 -- 172.21.15.99:0/2072589101 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4057070 con 0x558221d399e0 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5dfaa00640 1 -- 172.21.15.99:0/2072589101 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df402c980 con 0x558221d399e0 2024-03-23T23:46:06.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.119+0000 7f5dfaa00640 1 -- 172.21.15.99:0/2072589101 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4052590 con 0x558221d399e0 2024-03-23T23:46:06.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2072589101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 msgr2=0x558221d49f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x558221d34b70 tx=0x7f5df4000bc0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2072589101 shutdown_connections 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d3a3a0 0x558221d3d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d399e0 0x558221d49f00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/2072589101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d3cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.120+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2072589101 >> 172.21.15.99:0/2072589101 conn(0x558221d33c00 msgr2=0x558221d376f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.121+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2072589101 shutdown_connections 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.121+0000 7f5e13bd4740 1 -- 172.21.15.99:0/2072589101 wait complete. 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.121+0000 7f5e13bd4740 1 -- start start 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.121+0000 7f5e13bd4740 1 Processor -- start 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 0x558221d48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d3a3a0 0x558221d48880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1abef0 con 0x558221d38810 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1ab890 con 0x558221d399e0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1a8d90 con 0x558221d3a3a0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e13bd4740 10 client.0 _ll_register_callbacks cb 0x7f5dfc00eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e0a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d3a3a0 0x558221d48880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38132/0 (socket says 172.21.15.99:38132) 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 -- 172.21.15.99:0/1896155140 learned_addr learned my addr 172.21.15.99:0/1896155140 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09400640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 0x558221d48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 -- 172.21.15.99:0/1896155140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d3a3a0 msgr2=0x558221d48880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d3a3a0 0x558221d48880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 -- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 msgr2=0x558221d48340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 0x558221d48340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09e00640 1 -- 172.21.15.99:0/1896155140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558221d48fc0 con 0x558221d38810 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.122+0000 7f5e09400640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 0x558221d48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5e09e00640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0983a0 tx=0x7f5dfc086460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa05e6e1aadcdd46 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5defe00640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09b410 con 0x558221d38810 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5dfc0105d0 con 0x558221d38810 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df402c980 con 0x558221d38810 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc09bab0 con 0x558221d38810 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5e13bd4740 10 client.15915 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558221daac20 con 0x558221d38810 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.123+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df402c980 con 0x558221d38810 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f5dfc0b4c00 con 0x558221d38810 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 10 client.15915 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 20 client.15915 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221dab4a0 0x558221dad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 4 client.15915 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558221ca4b50 con 0x558221dab4a0 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.124+0000 7f5e09400640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221dab4a0 0x558221dad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.125+0000 7f5defe00640 10 client.15915 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:06.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.125+0000 7f5e09400640 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221dab4a0 0x558221dad8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b7e34c4c7a3cae server_cookie=6942c6c89907c3ac in_seq=0 out_seq=0 2024-03-23T23:46:06.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5defe00640 1 -- 172.21.15.99:0/1896155140 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558221ca4b50 con 0x558221dab4a0 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5defe00640 10 client.15915 handle_command_reply: tid=0 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 client.15915 shutdown 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 msgr2=0x558221d49ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0983a0 tx=0x7f5dfc086460 comp rx=0 tx=0).stop 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 shutdown_connections 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x558221dab4a0 0x558221dad8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.126+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558221d3a3a0 0x558221d48880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558221d399e0 0x558221d48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 1 --2- 172.21.15.99:0/1896155140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558221d38810 0x558221d49ce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 >> 172.21.15.99:0/1896155140 conn(0x558221d33c00 msgr2=0x558221d32620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5defe00640 0 client.15915 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 shutdown_connections 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 1 -- 172.21.15.99:0/1896155140 wait complete. 2024-03-23T23:46:06.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e13bd4740 20 client.15915 trim_cache size 0 max 16384 2024-03-23T23:46:06.131 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 msgr2=0x7f5df00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 0x7f5df00a4d40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x558221d398e0 tx=0x7f5e040046c0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 msgr2=0x7f5e0c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.127+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00ebc0 tx=0x7f5dfc00c2d0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 shutdown_connections 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f5df00a2880 0x7f5df00a4d40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1010c0 0x7f5e0c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e0c100700 0x7f5e0c19a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 --2- 172.21.15.99:0/524541704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e0c0ff9a0 0x7f5e0c199c70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 >> 172.21.15.99:0/524541704 conn(0x7f5e0c0fb2d0 msgr2=0x7f5e0c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 shutdown_connections 2024-03-23T23:46:06.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:06.128+0000 7f5e10e00640 1 -- 172.21.15.99:0/524541704 wait complete. 2024-03-23T23:46:06.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:05 smithi149 ceph-mon[36083]: pgmap v466: 97 pgs: 97 active+clean; 148 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 837 KiB/s wr, 339 op/s 2024-03-23T23:46:06.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:05 smithi174 ceph-mon[37002]: pgmap v466: 97 pgs: 97 active+clean; 148 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 837 KiB/s wr, 339 op/s 2024-03-23T23:46:06.264 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-23T23:46:06.265 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-23T23:46:06.266 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-03-23T23:46:06.267 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-03-23T23:46:06.268 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-23T23:46:06.269 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-23T23:46:06.270 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-03-23T23:46:06.271 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-03-23T23:46:06.272 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-23T23:46:06.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-03-23T23:46:06.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-23T23:46:06.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-03-23T23:46:06.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-23T23:46:06.273 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-23T23:46:06.871 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-03-23T23:46:06.872 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-23T23:46:06.873 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-03-23T23:46:06.874 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-23T23:46:06.875 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-03-23T23:46:06.876 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-23T23:46:07.152 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T23:46:07.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:46:07.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T23:46:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 -- 172.21.15.99:0/4250025757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 msgr2=0x7f8778106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 --2- 172.21.15.99:0/4250025757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f8778106e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f876402bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 -- 172.21.15.99:0/4250025757 shutdown_connections 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 --2- 172.21.15.99:0/4250025757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 0x7f87781097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 --2- 172.21.15.99:0/4250025757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f8778106e40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 --2- 172.21.15.99:0/4250025757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 0x7f8778104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 -- 172.21.15.99:0/4250025757 >> 172.21.15.99:0/4250025757 conn(0x7f87780fbb30 msgr2=0x7f87780fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 -- 172.21.15.99:0/4250025757 shutdown_connections 2024-03-23T23:46:07.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.400+0000 7f877e200640 1 -- 172.21.15.99:0/4250025757 wait complete. 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.401+0000 7f877e200640 1 Processor -- start 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.401+0000 7f877e200640 1 -- start start 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 0x7f8778101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 0x7f8778100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877810d380 con 0x7f8778107380 2024-03-23T23:46:07.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877810d200 con 0x7f8778102120 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f877e200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877810d500 con 0x7f8778104a50 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8776a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8777400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 0x7f8778101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8776a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50906/0 (socket says 172.21.15.99:50906) 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8776a00640 1 -- 172.21.15.99:0/833032802 learned_addr learned my addr 172.21.15.99:0/833032802 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8777e00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 0x7f8778100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8776a00640 1 -- 172.21.15.99:0/833032802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 msgr2=0x7f8778101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.402+0000 7f8776a00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 0x7f8778101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f8776a00640 1 -- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 msgr2=0x7f8778100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f8776a00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 0x7f8778100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f8776a00640 1 -- 172.21.15.99:0/833032802 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778100ea0 con 0x7f8778104a50 2024-03-23T23:46:07.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f8776a00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f876402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f876fe00640 1 -- 172.21.15.99:0/833032802 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764043020 con 0x7f8778104a50 2024-03-23T23:46:07.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f876fe00640 1 -- 172.21.15.99:0/833032802 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87640054e0 con 0x7f8778104a50 2024-03-23T23:46:07.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87781a1b40 con 0x7f8778104a50 2024-03-23T23:46:07.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.403+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87781a20e0 con 0x7f8778104a50 2024-03-23T23:46:07.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.404+0000 7f876fe00640 1 -- 172.21.15.99:0/833032802 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764003cd0 con 0x7f8778104a50 2024-03-23T23:46:07.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.407+0000 7f8780ffd740 1 Processor -- start 2024-03-23T23:46:07.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.407+0000 7f8780ffd740 1 -- start start 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.407+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf863f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.407+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf8644c00 0x55aaf8647090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8780ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf85a4eb0 con 0x55aaf863f0a0 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8780ffd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf854da60 con 0x55aaf8644c00 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8780ffd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf85a0f50 con 0x55aaf863fa90 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf863f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50914/0 (socket says 172.21.15.99:50914) 2024-03-23T23:46:07.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 -- 172.21.15.99:0/1254461351 learned_addr learned my addr 172.21.15.99:0/1254461351 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 -- 172.21.15.99:0/1254461351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf8644c00 msgr2=0x55aaf8647090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T23:46:07.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf8644c00 0x55aaf8647090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 -- 172.21.15.99:0/1254461351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 msgr2=0x55aaf863f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf863f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f8777400640 1 -- 172.21.15.99:0/1254461351 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf83817e0 con 0x55aaf863fa90 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.408+0000 7f876fe00640 1 -- 172.21.15.99:0/833032802 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8764046020 con 0x7f8778104a50 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.409+0000 7f8777400640 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8768009d20 tx=0x7f8768011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=159f2ae9b2f87d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.409+0000 7f8763e00640 1 -- 172.21.15.99:0/1254461351 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876801e070 con 0x55aaf863fa90 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.409+0000 7f8763e00640 1 -- 172.21.15.99:0/1254461351 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8768019b10 con 0x55aaf863fa90 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.409+0000 7f8763e00640 1 -- 172.21.15.99:0/1254461351 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800eb90 con 0x55aaf863fa90 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 -- 172.21.15.99:0/1254461351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 msgr2=0x55aaf8644590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8768009d20 tx=0x7f8768011810 comp rx=0 tx=0).stop 2024-03-23T23:46:07.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f876fe00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 0x7f875c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f876fe00640 1 -- 172.21.15.99:0/833032802 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87640f0a60 con 0x7f8778104a50 2024-03-23T23:46:07.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 -- 172.21.15.99:0/1254461351 shutdown_connections 2024-03-23T23:46:07.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf8644c00 0x55aaf8647090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863fa90 0x55aaf8644590 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1254461351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf863f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8780ffd740 1 -- 172.21.15.99:0/1254461351 >> 172.21.15.99:0/1254461351 conn(0x55aaf863a610 msgr2=0x55aaf863aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.410+0000 7f8777e00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 0x7f875c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.411+0000 7f8777e00640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 0x7f875c0a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55aaf86402f0 tx=0x7f8770009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.411+0000 7f8780ffd740 1 -- 172.21.15.99:0/1254461351 shutdown_connections 2024-03-23T23:46:07.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.412+0000 7f8780ffd740 1 -- 172.21.15.99:0/1254461351 wait complete. 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.412+0000 7f8780ffd740 1 -- start start 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.412+0000 7f8780ffd740 1 Processor -- start 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 0x55aaf8651a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf8651fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf8644c00 0x55aaf8658ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8650330 con 0x55aaf8644c00 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf8560eb0 con 0x55aaf863fa90 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf86504b0 con 0x55aaf863f0a0 2024-03-23T23:46:07.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8780ffd740 10 client.0 _ll_register_callbacks cb 0x55aaf8555150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8777400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 0x55aaf8651a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8777400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 0x55aaf8651a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50922/0 (socket says 172.21.15.99:50922) 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8776a00640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf8651fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8777400640 1 -- 172.21.15.99:0/2611348968 learned_addr learned my addr 172.21.15.99:0/2611348968 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.413+0000 7f8777e00640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf8644c00 0x55aaf8658ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 -- 172.21.15.99:0/2611348968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 msgr2=0x55aaf8651a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 0x55aaf8651a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 -- 172.21.15.99:0/2611348968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf8644c00 msgr2=0x55aaf8658ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf8644c00 0x55aaf8658ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 -- 172.21.15.99:0/2611348968 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaf83e1f70 con 0x55aaf863fa90 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8776a00640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf8651fb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87640bd080 tx=0x7f87640bd210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852e505532810bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8760c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640035f0 con 0x55aaf863fa90 2024-03-23T23:46:07.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87640c1130 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8768019b10 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.414+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8764005690 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 10 client.26117 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aaf8279170 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8768019b10 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f87640dd0b0 con 0x55aaf863fa90 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 10 client.26117 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 20 client.26117 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf865d850 0x55aaf865fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 4 client.26117 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.415+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaf85b23f0 con 0x55aaf865d850 2024-03-23T23:46:07.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.416+0000 7f8777400640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf865d850 0x55aaf865fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.416+0000 7f8777400640 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf865d850 0x55aaf865fc70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e0f46f501dc9f4 server_cookie=a1d106c33ce4708a in_seq=0 out_seq=0 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.416+0000 7f8760c00640 10 client.26117 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8760c00640 1 -- 172.21.15.99:0/2611348968 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aaf85b23f0 con 0x55aaf865d850 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8760c00640 10 client.26117 handle_command_reply: tid=0 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 client.26117 shutdown 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 msgr2=0x55aaf8651fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf8651fb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f87640bd080 tx=0x7f87640bd210 comp rx=0 tx=0).stop 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 shutdown_connections 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf865d850 0x55aaf865fc70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf8644c00 0x55aaf8658ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf8651fb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 --2- 172.21.15.99:0/2611348968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf863f0a0 0x55aaf8651a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8760c00640 0 client.26117 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.418+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 >> 172.21.15.99:0/2611348968 conn(0x55aaf863a610 msgr2=0x55aaf8639830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.419+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 shutdown_connections 2024-03-23T23:46:07.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.419+0000 7f8780ffd740 1 -- 172.21.15.99:0/2611348968 wait complete. 2024-03-23T23:46:07.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.419+0000 7f8780ffd740 20 client.26117 trim_cache size 0 max 16384 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.439+0000 7f8780ffd740 1 Processor -- start 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 -- start start 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863f0a0 0x55aaf8664550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 0x55aaf8658fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a67e0 con 0x55aaf863fa90 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a8160 con 0x55aaf863f0a0 2024-03-23T23:46:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8780ffd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a6e40 con 0x55aaf8644c00 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8777400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863f0a0 0x55aaf8664550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8777e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8777e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50926/0 (socket says 172.21.15.99:50926) 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8777e00640 1 -- 172.21.15.99:0/867316382 learned_addr learned my addr 172.21.15.99:0/867316382 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.440+0000 7f8776a00640 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 0x55aaf8658fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8777e00640 1 -- 172.21.15.99:0/867316382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863f0a0 msgr2=0x55aaf8664550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8777e00640 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863f0a0 0x55aaf8664550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8777e00640 1 -- 172.21.15.99:0/867316382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 msgr2=0x55aaf8658fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8777e00640 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 0x55aaf8658fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8777e00640 1 -- 172.21.15.99:0/867316382 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf8635c20 con 0x55aaf8644c00 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.441+0000 7f8776a00640 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 0x55aaf8658fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:07.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.442+0000 7f8777e00640 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8770003e90 tx=0x7f8770011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a97c3ed5958ec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.443+0000 7f8763400640 1 -- 172.21.15.99:0/867316382 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877000f480 con 0x55aaf8644c00 2024-03-23T23:46:07.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.443+0000 7f8763400640 1 -- 172.21.15.99:0/867316382 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f877000fb00 con 0x55aaf8644c00 2024-03-23T23:46:07.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.443+0000 7f8763400640 1 -- 172.21.15.99:0/867316382 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877001dab0 con 0x55aaf8644c00 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 -- 172.21.15.99:0/867316382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 msgr2=0x55aaf8664a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8770003e90 tx=0x7f8770011f90 comp rx=0 tx=0).stop 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 -- 172.21.15.99:0/867316382 shutdown_connections 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8664a90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863fa90 0x55aaf8658fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 --2- 172.21.15.99:0/867316382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863f0a0 0x55aaf8664550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 -- 172.21.15.99:0/867316382 >> 172.21.15.99:0/867316382 conn(0x55aaf863a610 msgr2=0x55aaf863e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 -- 172.21.15.99:0/867316382 shutdown_connections 2024-03-23T23:46:07.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.444+0000 7f8780ffd740 1 -- 172.21.15.99:0/867316382 wait complete. 2024-03-23T23:46:07.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.445+0000 7f8780ffd740 1 -- start start 2024-03-23T23:46:07.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.445+0000 7f8780ffd740 1 Processor -- start 2024-03-23T23:46:07.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.445+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf8658d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.445+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8780ffd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8657900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8780ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a7b00 con 0x55aaf863f0a0 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8780ffd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a74a0 con 0x55aaf863fa90 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8780ffd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87781a49c0 con 0x55aaf8644c00 2024-03-23T23:46:07.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8780ffd740 10 client.0 _ll_register_callbacks cb 0x7f87640b9620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48242/0 (socket says 172.21.15.99:48242) 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 -- 172.21.15.99:0/1296767082 learned_addr learned my addr 172.21.15.99:0/1296767082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8777e00640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8657900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8777400640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf8658d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 -- 172.21.15.99:0/1296767082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 msgr2=0x55aaf8657900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8657900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 -- 172.21.15.99:0/1296767082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 msgr2=0x55aaf8658d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf8658d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8776a00640 1 -- 172.21.15.99:0/1296767082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaf863b080 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.446+0000 7f8777e00640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8657900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8776a00640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f87640c56b0 tx=0x7f87640b9730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e627e042ef728cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8760c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640bcce0 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87640c1130 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f877000fb00 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87640035f0 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8780ffd740 10 client.26123 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aaf8650b30 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.447+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f877000fb00 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f87640bc840 con 0x55aaf863fa90 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 10 client.26123 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:07.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 20 client.26123 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf8648b20 0x55aaf864af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 4 client.26123 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55aaf85ac740 con 0x55aaf8648b20 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.448+0000 7f8777400640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf8648b20 0x55aaf864af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.449+0000 7f8777400640 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf8648b20 0x55aaf864af40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7db4cda3dd152 server_cookie=8d426b1b520464e4 in_seq=0 out_seq=0 2024-03-23T23:46:07.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.449+0000 7f8760c00640 10 client.26123 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8760c00640 1 -- 172.21.15.99:0/1296767082 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aaf85ac740 con 0x55aaf8648b20 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8760c00640 10 client.26123 handle_command_reply: tid=0 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 client.26123 shutdown 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 msgr2=0x55aaf86573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f87640c56b0 tx=0x7f87640b9730 comp rx=0 tx=0).stop 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 shutdown_connections 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x55aaf8648b20 0x55aaf864af40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaf8644c00 0x55aaf8657900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf863fa90 0x55aaf86573c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8760c00640 0 client.26123 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 --2- 172.21.15.99:0/1296767082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aaf863f0a0 0x55aaf8658d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.452+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 >> 172.21.15.99:0/1296767082 conn(0x55aaf863a610 msgr2=0x55aaf8640410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.453+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 shutdown_connections 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.453+0000 7f8780ffd740 1 -- 172.21.15.99:0/1296767082 wait complete. 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.453+0000 7f8780ffd740 20 client.26123 trim_cache size 0 max 16384 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "7dbb37fd-0095-46b6-9bc0-aad76aa354bd", 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-23T23:46:07.454 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:07.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 msgr2=0x7f875c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 0x7f875c0a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55aaf86402f0 tx=0x7f8770009500 comp rx=0 tx=0).stop 2024-03-23T23:46:07.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 msgr2=0x7f87781000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c00 tx=0x7f876402c650 comp rx=0 tx=0).stop 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 shutdown_connections 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f875c0a2880 0x7f875c0a4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8778107380 0x7f8778100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8778104a50 0x7f87781000d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 --2- 172.21.15.99:0/833032802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8778102120 0x7f8778101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 >> 172.21.15.99:0/833032802 conn(0x7f87780fbb30 msgr2=0x7f8778102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 shutdown_connections 2024-03-23T23:46:07.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.454+0000 7f877e200640 1 -- 172.21.15.99:0/833032802 wait complete. 2024-03-23T23:46:07.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:46:07.474 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:46:07.611 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-03-23T23:46:07.611 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-03-23T23:46:07.612 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-23T23:46:07.613 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-03-23T23:46:07.614 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-03-23T23:46:07.615 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-23T23:46:07.616 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-03-23T23:46:07.617 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-23T23:46:07.617 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-03-23T23:46:07.617 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 -- 172.21.15.99:0/2351668151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 msgr2=0x7f8708076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 --2- 172.21.15.99:0/2351668151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f8708076480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009900 tx=0x7f86f802bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 -- 172.21.15.99:0/2351668151 shutdown_connections 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 --2- 172.21.15.99:0/2351668151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 0x7f87080784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 --2- 172.21.15.99:0/2351668151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 0x7f8708077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 --2- 172.21.15.99:0/2351668151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f8708076480 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 -- 172.21.15.99:0/2351668151 >> 172.21.15.99:0/2351668151 conn(0x7f87080fee50 msgr2=0x7f8708101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 -- 172.21.15.99:0/2351668151 shutdown_connections 2024-03-23T23:46:07.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.683+0000 7f870ec00640 1 -- 172.21.15.99:0/2351668151 wait complete. 2024-03-23T23:46:07.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 Processor -- start 2024-03-23T23:46:07.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 -- start start 2024-03-23T23:46:07.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 0x7f870819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 0x7f87081a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708113970 con 0x7f8708077c40 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081137f0 con 0x7f87080772a0 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.685+0000 7f870ec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708113af0 con 0x7f8708076060 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50958/0 (socket says 172.21.15.99:50958) 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 -- 172.21.15.99:0/554081729 learned_addr learned my addr 172.21.15.99:0/554081729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707400640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 0x7f870819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f870cc00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 0x7f87081a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 -- 172.21.15.99:0/554081729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 msgr2=0x7f870819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 0x7f870819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 -- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 msgr2=0x7f87081a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 0x7f87081a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.686+0000 7f8707e00640 1 -- 172.21.15.99:0/554081729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87081a5bf0 con 0x7f8708076060 2024-03-23T23:46:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f8707e00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009230 tx=0x7f86f802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f8704c00640 1 -- 172.21.15.99:0/554081729 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8041070 con 0x7f8708076060 2024-03-23T23:46:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f8704c00640 1 -- 172.21.15.99:0/554081729 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f8004450 con 0x7f8708076060 2024-03-23T23:46:07.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87081a5e80 con 0x7f8708076060 2024-03-23T23:46:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f8704c00640 1 -- 172.21.15.99:0/554081729 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8003e20 con 0x7f8708076060 2024-03-23T23:46:07.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.687+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87081a62e0 con 0x7f8708076060 2024-03-23T23:46:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.690+0000 7f87119a0740 1 Processor -- start 2024-03-23T23:46:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.690+0000 7f87119a0740 1 -- start start 2024-03-23T23:46:07.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.690+0000 7f8704c00640 1 -- 172.21.15.99:0/554081729 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f86f8044070 con 0x7f8708076060 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe04a250 con 0x561efe0e5870 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdff2a30 con 0x561efe0e4e80 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f87119a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe0462f0 con 0x561efe0ea9e0 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48264/0 (socket says 172.21.15.99:48264) 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 -- 172.21.15.99:0/1132926665 learned_addr learned my addr 172.21.15.99:0/1132926665 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f8707e00640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f8707400640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 -- 172.21.15.99:0/1132926665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 msgr2=0x561efe0ece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0ece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.691+0000 7f870cc00640 1 -- 172.21.15.99:0/1132926665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 msgr2=0x561efe0ea370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.692+0000 7f870cc00640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ea370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.692+0000 7f870cc00640 1 -- 172.21.15.99:0/1132926665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efdfdd690 con 0x561efe0e4e80 2024-03-23T23:46:07.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.692+0000 7f8707e00640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.692+0000 7f8704c00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 0x7f86ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f8704c00640 1 -- 172.21.15.99:0/554081729 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86f80f05c0 con 0x7f8708076060 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f870cc00640 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f87000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7275956b4174f13a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f86fd200640 1 -- 172.21.15.99:0/1132926665 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700004d20 con 0x561efe0e4e80 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f86fd200640 1 -- 172.21.15.99:0/1132926665 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8700004050 con 0x561efe0e4e80 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f86fd200640 1 -- 172.21.15.99:0/1132926665 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700005260 con 0x561efe0e4e80 2024-03-23T23:46:07.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.693+0000 7f8707e00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 0x7f86ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 -- 172.21.15.99:0/1132926665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 msgr2=0x561efe0e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f87000118d0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f8707e00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 0x7f86ec0a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86f80ee040 tx=0x7f86f80f6060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 -- 172.21.15.99:0/1132926665 shutdown_connections 2024-03-23T23:46:07.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 --2- 172.21.15.99:0/1132926665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0e52a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.694+0000 7f87119a0740 1 -- 172.21.15.99:0/1132926665 >> 172.21.15.99:0/1132926665 conn(0x561efe0ad200 msgr2=0x561efe0e07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.695+0000 7f87119a0740 1 -- 172.21.15.99:0/1132926665 shutdown_connections 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.695+0000 7f87119a0740 1 -- 172.21.15.99:0/1132926665 wait complete. 2024-03-23T23:46:07.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.695+0000 7f87119a0740 1 -- start start 2024-03-23T23:46:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 Processor -- start 2024-03-23T23:46:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe0ed990 con 0x561efe0e5870 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe005e80 con 0x561efe0e4e80 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe0edb10 con 0x561efe0ea9e0 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.696+0000 7f87119a0740 10 client.0 _ll_register_callbacks cb 0x561efe018130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48270/0 (socket says 172.21.15.99:48270) 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 -- 172.21.15.99:0/2474319855 learned_addr learned my addr 172.21.15.99:0/2474319855 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 -- 172.21.15.99:0/2474319855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 msgr2=0x561efe0f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 -- 172.21.15.99:0/2474319855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 msgr2=0x561efe0ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 -- 172.21.15.99:0/2474319855 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efde287d0 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f870cc00640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8700009de0 tx=0x7f87000048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e3412cb20b382a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.697+0000 7f86f1600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000f4e0 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f870000f680 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8700004050 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f870000f880 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f87119a0740 10 client.26135 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.698+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561efdd5e3f0 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8700004050 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8700025040 con 0x561efe0e4e80 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 10 client.26135 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:07.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 20 client.26135 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe0faa10 0x561efe0fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 4 client.26135 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.699+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561efe057790 con 0x561efe0faa10 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.700+0000 7f8707400640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe0faa10 0x561efe0fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.700+0000 7f8707400640 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe0faa10 0x561efe0fce20 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d014080065f8aff server_cookie=7b2b25512e9344a5 in_seq=0 out_seq=0 2024-03-23T23:46:07.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.700+0000 7f86f1600640 10 client.26135 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.701+0000 7f86f1600640 1 -- 172.21.15.99:0/2474319855 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561efe057790 con 0x561efe0faa10 2024-03-23T23:46:07.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.701+0000 7f86f1600640 10 client.26135 handle_command_reply: tid=0 2024-03-23T23:46:07.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.701+0000 7f87119a0740 1 client.26135 shutdown 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 msgr2=0x561efe0eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8700009de0 tx=0x7f87000048e0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 shutdown_connections 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe0faa10 0x561efe0fce20 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f86f1600640 0 client.26135 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e5870 0x561efe0ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 --2- 172.21.15.99:0/2474319855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e4e80 0x561efe0eee50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.702+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 >> 172.21.15.99:0/2474319855 conn(0x561efe0ad200 msgr2=0x561efe0dfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.703+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 shutdown_connections 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.703+0000 7f87119a0740 1 -- 172.21.15.99:0/2474319855 wait complete. 2024-03-23T23:46:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.703+0000 7f87119a0740 20 client.26135 trim_cache size 0 max 16384 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 Processor -- start 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 -- start start 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 0x561efe101c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081af0a0 con 0x561efe0e4e80 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081b0a20 con 0x561efe0ea9e0 2024-03-23T23:46:07.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f87119a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081af700 con 0x561efe0e5870 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.726+0000 7f8707400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50994/0 (socket says 172.21.15.99:50994) 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 -- 172.21.15.99:0/3591383775 learned_addr learned my addr 172.21.15.99:0/3591383775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f870cc00640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f870cc00640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 0x561efe101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 -- 172.21.15.99:0/3591383775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 msgr2=0x561efe101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 0x561efe101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 -- 172.21.15.99:0/3591383775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 msgr2=0x561efe1016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe1016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 -- 172.21.15.99:0/3591383775 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efdd1e0f0 con 0x561efe0e5870 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f870cc00640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 0x561efe101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f8707400640 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x561efe0fd690 tx=0x7f86f40108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9525aacd637da5b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.727+0000 7f86f3e00640 1 -- 172.21.15.99:0/3591383775 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4004230 con 0x561efe0e5870 2024-03-23T23:46:07.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f86f3e00640 1 -- 172.21.15.99:0/3591383775 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f40043d0 con 0x561efe0e5870 2024-03-23T23:46:07.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f86f3e00640 1 -- 172.21.15.99:0/3591383775 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003b20 con 0x561efe0e5870 2024-03-23T23:46:07.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 -- 172.21.15.99:0/3591383775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 msgr2=0x561efe0f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x561efe0fd690 tx=0x7f86f40108c0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 -- 172.21.15.99:0/3591383775 shutdown_connections 2024-03-23T23:46:07.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0ea9e0 0x561efe101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0e5870 0x561efe0f6360 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 --2- 172.21.15.99:0/3591383775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.728+0000 7f87119a0740 1 -- 172.21.15.99:0/3591383775 >> 172.21.15.99:0/3591383775 conn(0x561efe0ad200 msgr2=0x561efe0e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.729+0000 7f87119a0740 1 -- 172.21.15.99:0/3591383775 shutdown_connections 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.729+0000 7f87119a0740 1 -- 172.21.15.99:0/3591383775 wait complete. 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.729+0000 7f87119a0740 1 -- start start 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.729+0000 7f87119a0740 1 Processor -- start 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe0f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081b03c0 con 0x561efe0e4e80 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081afd60 con 0x561efe0e5870 2024-03-23T23:46:07.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081ad260 con 0x561efe0ea9e0 2024-03-23T23:46:07.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f87119a0740 10 client.0 _ll_register_callbacks cb 0x7f86f80f3ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:07.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f8707400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f870cc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe0f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f8707400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:48284/0 (socket says 172.21.15.99:48284) 2024-03-23T23:46:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f8707400640 1 -- 172.21.15.99:0/2796300662 learned_addr learned my addr 172.21.15.99:0/2796300662 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.730+0000 7f870cc00640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 -- 172.21.15.99:0/2796300662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 msgr2=0x561efe0f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 -- 172.21.15.99:0/2796300662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 msgr2=0x561efe0f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe0f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 -- 172.21.15.99:0/2796300662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efe0f5b80 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f870cc00640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f8707400640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009cd0 tx=0x7f86f4005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcfcd790836f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f86f1600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f401db30 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86f401dcd0 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.731+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86f40043d0 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f401ded0 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 10 client.26138 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561efe156850 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f40043d0 con 0x561efe0e5870 2024-03-23T23:46:07.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f86f4025700 con 0x561efe0e5870 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 10 client.26138 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 20 client.26138 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe1571a0 0x561efe1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 4 client.26138 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.732+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561efe051ae0 con 0x561efe1571a0 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.733+0000 7f870cc00640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe1571a0 0x561efe1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:07.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.733+0000 7f870cc00640 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe1571a0 0x561efe1595c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f557a841c67436e0 server_cookie=9a4ac3915bb2665e in_seq=0 out_seq=0 2024-03-23T23:46:07.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.733+0000 7f86f1600640 10 client.26138 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.734+0000 7f86f1600640 1 -- 172.21.15.99:0/2796300662 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561efe051ae0 con 0x561efe1571a0 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.734+0000 7f86f1600640 10 client.26138 handle_command_reply: tid=0 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.734+0000 7f87119a0740 1 client.26138 shutdown 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 msgr2=0x561efe0f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009cd0 tx=0x7f86f4005790 comp rx=0 tx=0).stop 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 shutdown_connections 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x561efe1571a0 0x561efe1595c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561efe0ea9e0 0x561efe0f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe0e5870 0x561efe0f4580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 --2- 172.21.15.99:0/2796300662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561efe0e4e80 0x561efe0f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 >> 172.21.15.99:0/2796300662 conn(0x561efe0ad200 msgr2=0x561efe0e3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f86f1600640 0 client.26138 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 shutdown_connections 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 1 -- 172.21.15.99:0/2796300662 wait complete. 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.735+0000 7f87119a0740 20 client.26138 trim_cache size 0 max 16384 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:07.743 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: "7dbb37fd-0095-46b6-9bc0-aad76aa354bd": { 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "7dbb37fd-0095-46b6-9bc0-aad76aa354bd", 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:07.744 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 msgr2=0x7f86ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 0x7f86ec0a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86f80ee040 tx=0x7f86f80f6060 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 msgr2=0x7f870819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009230 tx=0x7f86f802c650 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 shutdown_connections 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f86ec0a2880 0x7f86ec0a4d40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8708077c40 0x7f87081a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87080772a0 0x7f870819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 --2- 172.21.15.99:0/554081729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708076060 0x7f870819dfd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 >> 172.21.15.99:0/554081729 conn(0x7f87080fee50 msgr2=0x7f87081021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 shutdown_connections 2024-03-23T23:46:07.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:07.736+0000 7f870ec00640 1 -- 172.21.15.99:0/554081729 wait complete. 2024-03-23T23:46:07.756 INFO:tasks.cephfs.filesystem:scrub status for tag:7dbb37fd-0095-46b6-9bc0-aad76aa354bd - {'path': '/', 'tag': '7dbb37fd-0095-46b6-9bc0-aad76aa354bd', 'options': 'recursive,force'} 2024-03-23T23:46:07.842 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:07 smithi099 ceph-mon[31715]: pgmap v467: 97 pgs: 97 active+clean; 192 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 863 KiB/s wr, 379 op/s 2024-03-23T23:46:08.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:07 smithi149 ceph-mon[36083]: pgmap v467: 97 pgs: 97 active+clean; 192 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 863 KiB/s wr, 379 op/s 2024-03-23T23:46:08.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:07 smithi174 ceph-mon[37002]: pgmap v467: 97 pgs: 97 active+clean; 192 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 89 MiB/s rd, 863 KiB/s wr, 379 op/s 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-23T23:46:08.822 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-23T23:46:08.823 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-23T23:46:08.824 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-03-23T23:46:08.825 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-03-23T23:46:08.826 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-23T23:46:08.827 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-03-23T23:46:09.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:08 smithi099 ceph-mon[31715]: scrub queued for path: / 2024-03-23T23:46:09.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:08 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:08 smithi099 ceph-mon[31715]: scrub summary: active paths [/] 2024-03-23T23:46:09.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:08 smithi099 ceph-mon[31715]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:08 smithi099 ceph-mon[31715]: scrub summary: idle 2024-03-23T23:46:09.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:08 smithi149 ceph-mon[36083]: scrub queued for path: / 2024-03-23T23:46:09.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:08 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:08 smithi149 ceph-mon[36083]: scrub summary: active paths [/] 2024-03-23T23:46:09.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:08 smithi149 ceph-mon[36083]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:08 smithi149 ceph-mon[36083]: scrub summary: idle 2024-03-23T23:46:09.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:08 smithi174 ceph-mon[37002]: scrub queued for path: / 2024-03-23T23:46:09.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:08 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:08 smithi174 ceph-mon[37002]: scrub summary: active paths [/] 2024-03-23T23:46:09.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:08 smithi174 ceph-mon[37002]: scrub summary: idle+waiting paths [/] 2024-03-23T23:46:09.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:08 smithi174 ceph-mon[37002]: scrub summary: idle 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-23T23:46:09.828 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-23T23:46:09.829 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-03-23T23:46:09.830 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-23T23:46:09.831 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-23T23:46:09.832 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-03-23T23:46:09.833 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-23T23:46:10.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:09 smithi099 ceph-mon[31715]: pgmap v468: 97 pgs: 97 active+clean; 171 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 914 KiB/s wr, 379 op/s 2024-03-23T23:46:10.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:09 smithi149 ceph-mon[36083]: pgmap v468: 97 pgs: 97 active+clean; 171 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 914 KiB/s wr, 379 op/s 2024-03-23T23:46:10.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:09 smithi174 ceph-mon[37002]: pgmap v468: 97 pgs: 97 active+clean; 171 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 914 KiB/s wr, 379 op/s 2024-03-23T23:46:10.370 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-03-23T23:46:10.370 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-23T23:46:10.370 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-03-23T23:46:10.370 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-03-23T23:46:10.371 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-23T23:46:10.372 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-03-23T23:46:10.373 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-23T23:46:10.374 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-03-23T23:46:10.375 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-23T23:46:10.376 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-03-23T23:46:10.376 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-23T23:46:10.376 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-23T23:46:11.406 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-03-23T23:46:11.407 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-23T23:46:11.408 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-23T23:46:11.409 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-03-23T23:46:11.410 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-03-23T23:46:11.411 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-23T23:46:12.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:11 smithi099 ceph-mon[31715]: pgmap v469: 97 pgs: 97 active+clean; 151 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 836 KiB/s wr, 374 op/s 2024-03-23T23:46:12.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:11 smithi149 ceph-mon[36083]: pgmap v469: 97 pgs: 97 active+clean; 151 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 836 KiB/s wr, 374 op/s 2024-03-23T23:46:12.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:11 smithi174 ceph-mon[37002]: pgmap v469: 97 pgs: 97 active+clean; 151 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 836 KiB/s wr, 374 op/s 2024-03-23T23:46:12.745 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-03-23T23:46:12.745 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-23T23:46:12.745 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-03-23T23:46:12.745 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-03-23T23:46:12.746 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-03-23T23:46:12.747 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-23T23:46:12.748 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-03-23T23:46:12.749 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-23T23:46:12.750 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-03-23T23:46:12.751 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-23T23:46:12.751 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-03-23T23:46:12.751 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-23T23:46:12.751 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-03-23T23:46:12.751 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-23T23:46:13.488 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-03-23T23:46:13.489 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-23T23:46:13.490 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-23T23:46:13.491 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-23T23:46:13.492 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-23T23:46:13.493 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-23T23:46:13.494 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-03-23T23:46:14.128 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-03-23T23:46:14.129 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-23T23:46:14.130 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-23T23:46:14.131 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-23T23:46:14.132 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-03-23T23:46:14.133 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-03-23T23:46:14.134 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-23T23:46:14.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:13 smithi149 ceph-mon[36083]: pgmap v470: 97 pgs: 97 active+clean; 115 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 777 KiB/s wr, 349 op/s 2024-03-23T23:46:14.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:13 smithi174 ceph-mon[37002]: pgmap v470: 97 pgs: 97 active+clean; 115 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 777 KiB/s wr, 349 op/s 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-23T23:46:14.222 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-03-23T23:46:14.223 INFO:tasks.workunit.client.0.smithi099.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-23T23:46:14.224 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T23:46:14.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T23:46:14.303 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:13 smithi099 ceph-mon[31715]: pgmap v470: 97 pgs: 97 active+clean; 115 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 777 KiB/s wr, 349 op/s 2024-03-23T23:46:14.322 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-03-23T23:46:14.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-23T23:46:15.498 DEBUG:teuthology.parallel:result is None 2024-03-23T23:46:15.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T23:46:15.586 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T23:46:15.586 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-23T23:46:15.597 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:46:15.617 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-23T23:46:15.682 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-23T23:46:16.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:15 smithi149 ceph-mon[36083]: pgmap v471: 97 pgs: 97 active+clean; 153 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 989 KiB/s wr, 398 op/s 2024-03-23T23:46:16.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:15 smithi174 ceph-mon[37002]: pgmap v471: 97 pgs: 97 active+clean; 153 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 989 KiB/s wr, 398 op/s 2024-03-23T23:46:16.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:15 smithi099 ceph-mon[31715]: pgmap v471: 97 pgs: 97 active+clean; 153 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 989 KiB/s wr, 398 op/s 2024-03-23T23:46:18.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:17 smithi149 ceph-mon[36083]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 800 KiB/s wr, 355 op/s 2024-03-23T23:46:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:17 smithi174 ceph-mon[37002]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 800 KiB/s wr, 355 op/s 2024-03-23T23:46:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:17 smithi099 ceph-mon[31715]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 800 KiB/s wr, 355 op/s 2024-03-23T23:46:20.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:19 smithi149 ceph-mon[36083]: pgmap v473: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 807 KiB/s wr, 309 op/s 2024-03-23T23:46:20.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:19 smithi174 ceph-mon[37002]: pgmap v473: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 807 KiB/s wr, 309 op/s 2024-03-23T23:46:20.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:19 smithi099 ceph-mon[31715]: pgmap v473: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 807 KiB/s wr, 309 op/s 2024-03-23T23:46:22.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:21 smithi149 ceph-mon[36083]: pgmap v474: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 475 KiB/s wr, 207 op/s 2024-03-23T23:46:22.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:21 smithi174 ceph-mon[37002]: pgmap v474: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 475 KiB/s wr, 207 op/s 2024-03-23T23:46:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:21 smithi099 ceph-mon[31715]: pgmap v474: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 475 KiB/s wr, 207 op/s 2024-03-23T23:46:24.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:23 smithi149 ceph-mon[36083]: pgmap v475: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 416 KiB/s wr, 166 op/s 2024-03-23T23:46:24.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:23 smithi174 ceph-mon[37002]: pgmap v475: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 416 KiB/s wr, 166 op/s 2024-03-23T23:46:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:23 smithi099 ceph-mon[31715]: pgmap v475: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 416 KiB/s wr, 166 op/s 2024-03-23T23:46:26.145 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:25 smithi149 ceph-mon[36083]: pgmap v476: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 376 KiB/s wr, 146 op/s 2024-03-23T23:46:26.164 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:25 smithi099 ceph-mon[31715]: pgmap v476: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 376 KiB/s wr, 146 op/s 2024-03-23T23:46:26.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:25 smithi174 ceph-mon[37002]: pgmap v476: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 376 KiB/s wr, 146 op/s 2024-03-23T23:46:28.144 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:27 smithi149 ceph-mon[36083]: pgmap v477: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 127 KiB/s wr, 76 op/s 2024-03-23T23:46:28.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:27 smithi174 ceph-mon[37002]: pgmap v477: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 127 KiB/s wr, 76 op/s 2024-03-23T23:46:28.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:27 smithi099 ceph-mon[31715]: pgmap v477: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 127 KiB/s wr, 76 op/s 2024-03-23T23:46:30.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:29 smithi174 ceph-mon[37002]: pgmap v478: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 78 KiB/s wr, 43 op/s 2024-03-23T23:46:30.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:29 smithi099 ceph-mon[31715]: pgmap v478: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 78 KiB/s wr, 43 op/s 2024-03-23T23:46:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:29 smithi149 ceph-mon[36083]: pgmap v478: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 78 KiB/s wr, 43 op/s 2024-03-23T23:46:32.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:31 smithi174 ceph-mon[37002]: pgmap v479: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-03-23T23:46:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:31 smithi099 ceph-mon[31715]: pgmap v479: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-03-23T23:46:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:31 smithi149 ceph-mon[36083]: pgmap v479: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-03-23T23:46:34.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:33 smithi174 ceph-mon[37002]: pgmap v480: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:46:34.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:33 smithi099 ceph-mon[31715]: pgmap v480: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:46:34.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:33 smithi149 ceph-mon[36083]: pgmap v480: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-23T23:46:36.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:35 smithi174 ceph-mon[37002]: pgmap v481: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-03-23T23:46:36.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:35 smithi099 ceph-mon[31715]: pgmap v481: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-03-23T23:46:36.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:35 smithi149 ceph-mon[36083]: pgmap v481: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1023 B/s wr, 12 op/s 2024-03-23T23:46:37.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T23:46:37.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T23:46:37.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 -- 172.21.15.99:0/2979980614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041028a0 msgr2=0x7fd504102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 --2- 172.21.15.99:0/2979980614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041028a0 0x7fd504102d20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009900 tx=0x7fd4ec02bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 -- 172.21.15.99:0/2979980614 shutdown_connections 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 --2- 172.21.15.99:0/2979980614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd504103260 0x7fd504109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 --2- 172.21.15.99:0/2979980614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041028a0 0x7fd504102d20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 --2- 172.21.15.99:0/2979980614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5041016b0 0x7fd504101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 -- 172.21.15.99:0/2979980614 >> 172.21.15.99:0/2979980614 conn(0x7fd5040fcc40 msgr2=0x7fd5040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 -- 172.21.15.99:0/2979980614 shutdown_connections 2024-03-23T23:46:37.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.990+0000 7fd50ac00640 1 -- 172.21.15.99:0/2979980614 wait complete. 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.991+0000 7fd50ac00640 1 Processor -- start 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.991+0000 7fd50ac00640 1 -- start start 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5041028a0 0x7fd50419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd504103260 0x7fd5041a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50410d380 con 0x7fd5041028a0 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50410d200 con 0x7fd504103260 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd50ac00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50410d500 con 0x7fd5041016b0 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56420/0 (socket says 172.21.15.99:56420) 2024-03-23T23:46:37.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 -- 172.21.15.99:0/3098370104 learned_addr learned my addr 172.21.15.99:0/3098370104 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd508c00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd504103260 0x7fd5041a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5041028a0 0x7fd50419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 -- 172.21.15.99:0/3098370104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd504103260 msgr2=0x7fd5041a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd504103260 0x7fd5041a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 -- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5041028a0 msgr2=0x7fd50419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5041028a0 0x7fd50419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.992+0000 7fd503e00640 1 -- 172.21.15.99:0/3098370104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5041a1810 con 0x7fd5041016b0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd503e00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400e9c0 tx=0x7fd4f400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd501600640 1 -- 172.21.15.99:0/3098370104 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400cd80 con 0x7fd5041016b0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd501600640 1 -- 172.21.15.99:0/3098370104 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f40035f0 con 0x7fd5041016b0 2024-03-23T23:46:37.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041a1b00 con 0x7fd5041016b0 2024-03-23T23:46:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd501600640 1 -- 172.21.15.99:0/3098370104 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4010690 con 0x7fd5041016b0 2024-03-23T23:46:37.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.993+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041a1fd0 con 0x7fd5041016b0 2024-03-23T23:46:37.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.996+0000 7fd501600640 1 -- 172.21.15.99:0/3098370104 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd4f40037a0 con 0x7fd5041016b0 2024-03-23T23:46:37.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 Processor -- start 2024-03-23T23:46:37.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd501600640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:37.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 -- start start 2024-03-23T23:46:37.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd501600640 1 -- 172.21.15.99:0/3098370104 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd4f40cee50 con 0x7fd5041016b0 2024-03-23T23:46:37.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd508c00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:37.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321561370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x560321563e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603214c1250 con 0x56032155c870 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560321469a30 con 0x5603215619e0 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.998+0000 7fd50d830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603214bd2f0 con 0x56032155be80 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56430/0 (socket says 172.21.15.99:56430) 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 -- 172.21.15.99:0/3359876205 learned_addr learned my addr 172.21.15.99:0/3359876205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321561370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 -- 172.21.15.99:0/3359876205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 msgr2=0x560321563e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x560321563e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 -- 172.21.15.99:0/3359876205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 msgr2=0x560321561370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321561370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 -- 172.21.15.99:0/3359876205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560321454690 con 0x56032155be80 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd503e00640 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40a39e0 tx=0x7fd4f409b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6b3d12214371b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd508c00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc005fd0 tx=0x7fd4fc009b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd4f9200640 1 -- 172.21.15.99:0/3359876205 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40b4070 con 0x56032155be80 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd4f9200640 1 -- 172.21.15.99:0/3359876205 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f409f8c0 con 0x56032155be80 2024-03-23T23:46:38.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:37.999+0000 7fd4f9200640 1 -- 172.21.15.99:0/3359876205 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40a3a80 con 0x56032155be80 2024-03-23T23:46:38.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 -- 172.21.15.99:0/3359876205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 msgr2=0x56032155c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40a39e0 tx=0x7fd4f409b0d0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 -- 172.21.15.99:0/3359876205 shutdown_connections 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x560321563e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321561370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 --2- 172.21.15.99:0/3359876205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x56032155c2a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 -- 172.21.15.99:0/3359876205 >> 172.21.15.99:0/3359876205 conn(0x560321524200 msgr2=0x5603215577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 -- 172.21.15.99:0/3359876205 shutdown_connections 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.000+0000 7fd50d830740 1 -- 172.21.15.99:0/3359876205 wait complete. 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 -- start start 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 Processor -- start 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321566330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032156d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560321564930 con 0x56032155c870 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032147ce80 con 0x5603215619e0 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560321564ab0 con 0x56032155be80 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd50d830740 10 client.0 _ll_register_callbacks cb 0x56032148f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.001+0000 7fd503e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56432/0 (socket says 172.21.15.99:56432) 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 -- 172.21.15.99:0/118890033 learned_addr learned my addr 172.21.15.99:0/118890033 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd508c00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032156d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321566330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 -- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 msgr2=0x56032156d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032156d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 -- 172.21.15.99:0/118890033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 msgr2=0x560321566330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321566330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 -- 172.21.15.99:0/118890033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56032129f7d0 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321566330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd503e00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4097bd0 tx=0x7fd4f4003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177d457764c294ca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd4f1600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f407f060 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd4f4099810 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.002+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4f409f8c0 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f40b8030 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 10 client.26182 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603211d53f0 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f409f8c0 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd4f409abc0 con 0x56032155be80 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 10 client.26182 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:38.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 20 client.26182 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x560321571860 0x560321573c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 4 client.26182 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.003+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603214ce790 con 0x560321571860 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.004+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x560321571860 0x560321573c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.004+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x560321571860 0x560321573c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86eed9e93ab95b0 server_cookie=68e05f382196e202 in_seq=0 out_seq=0 2024-03-23T23:46:38.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.004+0000 7fd4f1600640 10 client.26182 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd4f1600640 1 -- 172.21.15.99:0/118890033 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5603214ce790 con 0x560321571860 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd4f1600640 10 client.26182 handle_command_reply: tid=0 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 client.26182 shutdown 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 msgr2=0x560321565df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4097bd0 tx=0x7fd4f4003d70 comp rx=0 tx=0).stop 2024-03-23T23:46:38.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 shutdown_connections 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x560321571860 0x560321573c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032156d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155c870 0x560321566330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd4f1600640 0 client.26182 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 --2- 172.21.15.99:0/118890033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155be80 0x560321565df0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.005+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 >> 172.21.15.99:0/118890033 conn(0x560321524200 msgr2=0x560321556a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.006+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 shutdown_connections 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.006+0000 7fd50d830740 1 -- 172.21.15.99:0/118890033 wait complete. 2024-03-23T23:46:38.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.006+0000 7fd50d830740 20 client.26182 trim_cache size 0 max 16384 2024-03-23T23:46:38.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 Processor -- start 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 -- start start 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032155b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032155d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041aacb0 con 0x56032155be80 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041ac630 con 0x5603215619e0 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.039+0000 7fd50d830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041ab310 con 0x56032155c870 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56444/0 (socket says 172.21.15.99:56444) 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 -- 172.21.15.99:0/1863745118 learned_addr learned my addr 172.21.15.99:0/1863745118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd503e00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032155b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd508c00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032155d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 -- 172.21.15.99:0/1863745118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 msgr2=0x56032155d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032155d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 -- 172.21.15.99:0/1863745118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 msgr2=0x56032155b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032155b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 -- 172.21.15.99:0/1863745118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603211950f0 con 0x56032155c870 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd508c00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032155d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd503e00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032155b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.040+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005ab0 tx=0x7fd4ec02df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afb8a3b6474c1ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd4f3e00640 1 -- 172.21.15.99:0/1863745118 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec02cb20 con 0x56032155c870 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd4f3e00640 1 -- 172.21.15.99:0/1863745118 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4ec04f030 con 0x56032155c870 2024-03-23T23:46:38.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd4f3e00640 1 -- 172.21.15.99:0/1863745118 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec004eb0 con 0x56032155c870 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 -- 172.21.15.99:0/1863745118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 msgr2=0x56032156d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005ab0 tx=0x7fd4ec02df90 comp rx=0 tx=0).stop 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 -- 172.21.15.99:0/1863745118 shutdown_connections 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603215619e0 0x56032155d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032155c870 0x56032156d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 --2- 172.21.15.99:0/1863745118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032155b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 -- 172.21.15.99:0/1863745118 >> 172.21.15.99:0/1863745118 conn(0x560321524200 msgr2=0x56032155d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 -- 172.21.15.99:0/1863745118 shutdown_connections 2024-03-23T23:46:38.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.041+0000 7fd50d830740 1 -- 172.21.15.99:0/1863745118 wait complete. 2024-03-23T23:46:38.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.042+0000 7fd50d830740 1 -- start start 2024-03-23T23:46:38.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.042+0000 7fd50d830740 1 Processor -- start 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032156b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041abfd0 con 0x56032155be80 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041ab970 con 0x56032155c870 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a8e70 con 0x5603215619e0 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd50d830740 10 client.0 _ll_register_callbacks cb 0x7fd4f40b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd508c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd503e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032156b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd508c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56456/0 (socket says 172.21.15.99:56456) 2024-03-23T23:46:38.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd4fbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40238/0 (socket says 172.21.15.99:40238) 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd508c00640 1 -- 172.21.15.99:0/2237098976 learned_addr learned my addr 172.21.15.99:0/2237098976 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.043+0000 7fd508c00640 1 -- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 msgr2=0x56032156b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd508c00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd508c00640 1 -- 172.21.15.99:0/2237098976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 msgr2=0x56032156b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd508c00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032156b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd508c00640 1 -- 172.21.15.99:0/2237098976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56032156cd20 con 0x5603215619e0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd4fbe00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd503e00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032156b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd508c00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc003f80 tx=0x7fd4fc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0173a93a9b42122 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd4f1600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004b20 con 0x5603215619e0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd4fc004cc0 con 0x5603215619e0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.044+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4ec04f030 con 0x5603215619e0 2024-03-23T23:46:38.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4fc00f690 con 0x5603215619e0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 10 client.26188 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603215cd640 con 0x5603215619e0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4ec04f030 con 0x5603215619e0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fd4fc005410 con 0x5603215619e0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 10 client.26188 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 20 client.26188 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5603215ce050 0x5603215d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 4 client.26188 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5603214c8ae0 con 0x5603215ce050 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.045+0000 7fd503e00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5603215ce050 0x5603215d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd503e00640 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5603215ce050 0x5603215d0470 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfb1ef1416fa769 server_cookie=b83c4c651c888e5d in_seq=0 out_seq=0 2024-03-23T23:46:38.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd4f1600640 10 client.26188 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd4f1600640 1 -- 172.21.15.99:0/2237098976 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5603214c8ae0 con 0x5603215ce050 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd4f1600640 10 client.26188 handle_command_reply: tid=0 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 client.26188 shutdown 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 msgr2=0x56032156c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc003f80 tx=0x7fd4fc004ab0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 shutdown_connections 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5603215ce050 0x5603215d0470 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603215619e0 0x56032156c730 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd4f1600640 0 client.26188 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56032155c870 0x56032156b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 --2- 172.21.15.99:0/2237098976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56032155be80 0x56032156b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 >> 172.21.15.99:0/2237098976 conn(0x560321524200 msgr2=0x56032155b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 shutdown_connections 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 1 -- 172.21.15.99:0/2237098976 wait complete. 2024-03-23T23:46:38.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.046+0000 7fd50d830740 20 client.26188 trim_cache size 0 max 16384 2024-03-23T23:46:38.054 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-23T23:46:38.054 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-23T23:46:38.054 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 msgr2=0x7fd4e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc005fd0 tx=0x7fd4fc009b00 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 msgr2=0x7fd504199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400e9c0 tx=0x7fd4f400ee80 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 shutdown_connections 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd504103260 0x7fd5041a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5041028a0 0x7fd50419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 --2- 172.21.15.99:0/3098370104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5041016b0 0x7fd504199bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 >> 172.21.15.99:0/3098370104 conn(0x7fd5040fcc40 msgr2=0x7fd5040738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 shutdown_connections 2024-03-23T23:46:38.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.047+0000 7fd50ac00640 1 -- 172.21.15.99:0/3098370104 wait complete. 2024-03-23T23:46:38.072 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T23:46:38.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:46:38.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:37 smithi099 ceph-mon[31715]: pgmap v482: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:46:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:37 smithi174 ceph-mon[37002]: pgmap v482: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 -- 172.21.15.99:0/1123637088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281030b0 msgr2=0x7f6628109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 --2- 172.21.15.99:0/1123637088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281030b0 0x7f6628109940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6620009ca0 tx=0x7f662002c100 comp rx=0 tx=0).stop 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 -- 172.21.15.99:0/1123637088 shutdown_connections 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 --2- 172.21.15.99:0/1123637088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281030b0 0x7f6628109940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 --2- 172.21.15.99:0/1123637088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66281026f0 0x7f6628102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 --2- 172.21.15.99:0/1123637088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6628101500 0x7f6628101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 -- 172.21.15.99:0/1123637088 >> 172.21.15.99:0/1123637088 conn(0x7f66280fca70 msgr2=0x7f66280feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.266+0000 7f662ea00640 1 -- 172.21.15.99:0/1123637088 shutdown_connections 2024-03-23T23:46:38.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.267+0000 7f662ea00640 1 -- 172.21.15.99:0/1123637088 wait complete. 2024-03-23T23:46:38.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.268+0000 7f662ea00640 1 Processor -- start 2024-03-23T23:46:38.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.268+0000 7f662ea00640 1 -- start start 2024-03-23T23:46:38.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6628101500 0x7f6628075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 0x7f6628073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662810d190 con 0x7f6628101500 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662810d010 con 0x7f66281030b0 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662ea00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662810d310 con 0x7f66281026f0 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662cc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 0x7f6628073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662cc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56486/0 (socket says 172.21.15.99:56486) 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662cc00640 1 -- 172.21.15.99:0/2856270061 learned_addr learned my addr 172.21.15.99:0/2856270061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662d600640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6628101500 0x7f6628075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 -- 172.21.15.99:0/2856270061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 msgr2=0x7f6628073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 -- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6628101500 msgr2=0x7f6628075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6628101500 0x7f6628075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662e000640 1 -- 172.21.15.99:0/2856270061 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628074860 con 0x7f66281030b0 2024-03-23T23:46:38.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.269+0000 7f662cc00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:38.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f662e000640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 0x7f6628073fd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6620009bf0 tx=0x7f6620035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620039040 con 0x7f66281030b0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6620044070 con 0x7f66281030b0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66281a1870 con 0x7f66281030b0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200043c0 con 0x7f66281030b0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.270+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281a1de0 con 0x7f66281030b0 2024-03-23T23:46:38.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.272+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66281087c0 con 0x7f66281030b0 2024-03-23T23:46:38.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.275+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6620004560 con 0x7f66281030b0 2024-03-23T23:46:38.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.276+0000 7f6626000640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 0x7f66100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.276+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66200f0f70 con 0x7f66281030b0 2024-03-23T23:46:38.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.276+0000 7f662d600640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 0x7f66100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.277+0000 7f662d600640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 0x7f66100a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f661c0095f0 tx=0x7f661c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.282+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f66200bdff0 con 0x7f66281030b0 2024-03-23T23:46:38.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:37 smithi149 ceph-mon[36083]: pgmap v482: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:46:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.523+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66280622a0 con 0x7f66281030b0 2024-03-23T23:46:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.524+0000 7f6626000640 1 -- 172.21.15.99:0/2856270061 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f66200098b0 con 0x7f66281030b0 2024-03-23T23:46:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:46:38.524 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:46:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.526+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 msgr2=0x7f66100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.526+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 0x7f66100a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f661c0095f0 tx=0x7f661c005f90 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.526+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 msgr2=0x7f6628073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.526+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 0x7f6628073fd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6620009bf0 tx=0x7f6620035d10 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 shutdown_connections 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f66100a2880 0x7f66100a4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66281030b0 0x7f6628073fd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66281026f0 0x7f6628073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 --2- 172.21.15.99:0/2856270061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6628101500 0x7f6628075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 >> 172.21.15.99:0/2856270061 conn(0x7f66280fca70 msgr2=0x7f6628105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 shutdown_connections 2024-03-23T23:46:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.527+0000 7f662ea00640 1 -- 172.21.15.99:0/2856270061 wait complete. 2024-03-23T23:46:38.545 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:38.545 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 -- 172.21.15.99:0/550058251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450102790 msgr2=0x7f2450102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 --2- 172.21.15.99:0/550058251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450102790 0x7f2450102c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f243c009900 tx=0x7f243c02bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 -- 172.21.15.99:0/550058251 shutdown_connections 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 --2- 172.21.15.99:0/550058251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450103150 0x7f24501099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 --2- 172.21.15.99:0/550058251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450102790 0x7f2450102c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 --2- 172.21.15.99:0/550058251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f24501019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 -- 172.21.15.99:0/550058251 >> 172.21.15.99:0/550058251 conn(0x7f24500fcb10 msgr2=0x7f24500fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 -- 172.21.15.99:0/550058251 shutdown_connections 2024-03-23T23:46:38.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.818+0000 7f2456400640 1 -- 172.21.15.99:0/550058251 wait complete. 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.819+0000 7f2456400640 1 Processor -- start 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 -- start start 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450102790 0x7f2450073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450103150 0x7f2450073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245010d230 con 0x7f2450102790 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245010d0b0 con 0x7f24501015a0 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2456400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245010d3b0 con 0x7f2450103150 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2455000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2455000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40274/0 (socket says 172.21.15.99:40274) 2024-03-23T23:46:38.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2455000640 1 -- 172.21.15.99:0/943302652 learned_addr learned my addr 172.21.15.99:0/943302652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.820+0000 7f2455a00640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450103150 0x7f2450073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f244fe00640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450102790 0x7f2450073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 -- 172.21.15.99:0/943302652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450103150 msgr2=0x7f2450073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450103150 0x7f2450073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 -- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450102790 msgr2=0x7f2450073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450102790 0x7f2450073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 -- 172.21.15.99:0/943302652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450074860 con 0x7f24501015a0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2455000640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9c0 tx=0x7f244400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f244d600640 1 -- 172.21.15.99:0/943302652 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400cde0 con 0x7f24501015a0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24501a1980 con 0x7f24501015a0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f244d600640 1 -- 172.21.15.99:0/943302652 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2444010910 con 0x7f24501015a0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.821+0000 7f244d600640 1 -- 172.21.15.99:0/943302652 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440189a0 con 0x7f24501015a0 2024-03-23T23:46:38.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.822+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24501a1ea0 con 0x7f24501015a0 2024-03-23T23:46:38.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.824+0000 7f244d600640 1 -- 172.21.15.99:0/943302652 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f244401f020 con 0x7f24501015a0 2024-03-23T23:46:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.825+0000 7f245a627740 1 Processor -- start 2024-03-23T23:46:38.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.825+0000 7f245a627740 1 -- start start 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9694c2c0 con 0x55ee969e79f0 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee968f4c90 con 0x55ee969e9580 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f245a627740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee96948360 con 0x55ee969e8bc0 2024-03-23T23:46:38.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f244fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:56524/0 (socket says 172.21.15.99:56524) 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 -- 172.21.15.99:0/1843609359 learned_addr learned my addr 172.21.15.99:0/1843609359 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f244fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969efe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:40282/0 (socket says 172.21.15.99:40282) 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 -- 172.21.15.99:0/1843609359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 msgr2=0x55ee969efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 -- 172.21.15.99:0/1843609359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 msgr2=0x55ee969e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.826+0000 7f2455000640 1 -- 172.21.15.99:0/1843609359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee96919080 con 0x55ee969e8bc0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f244d600640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 0x7f24380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f2455000640 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24440d2ea0 tx=0x7f24440d2f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1234a30c32a699cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f244d600640 1 -- 172.21.15.99:0/943302652 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24440ced70 con 0x7f24501015a0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f2455a00640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 0x7f24380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f2442000640 1 -- 172.21.15.99:0/1843609359 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444097930 con 0x55ee969e8bc0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f2442000640 1 -- 172.21.15.99:0/1843609359 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24440860c0 con 0x55ee969e8bc0 2024-03-23T23:46:38.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.827+0000 7f2442000640 1 -- 172.21.15.99:0/1843609359 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444010ac0 con 0x55ee969e8bc0 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 -- 172.21.15.99:0/1843609359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 msgr2=0x55ee969e9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f2455a00640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 0x7f24380a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ee969e8ac0 tx=0x7f2448009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f24440d2ea0 tx=0x7f24440d2f90 comp rx=0 tx=0).stop 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 -- 172.21.15.99:0/1843609359 shutdown_connections 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969e9040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 --2- 172.21.15.99:0/1843609359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 -- 172.21.15.99:0/1843609359 >> 172.21.15.99:0/1843609359 conn(0x55ee969e2de0 msgr2=0x55ee969e31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 -- 172.21.15.99:0/1843609359 shutdown_connections 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.828+0000 7f245a627740 1 -- 172.21.15.99:0/1843609359 wait complete. 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 -- start start 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 Processor -- start 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee969f0590 con 0x55ee969e79f0 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee969080d0 con 0x55ee969e9580 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee969f0710 con 0x55ee969e8bc0 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f245a627740 10 client.0 _ll_register_callbacks cb 0x55ee9690ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:38.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f244fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.829+0000 7f2455a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f244fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57034/0 (socket says 172.21.15.99:57034) 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f244fe00640 1 -- 172.21.15.99:0/2298111498 learned_addr learned my addr 172.21.15.99:0/2298111498 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455000640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 -- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 msgr2=0x55ee969f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 -- 172.21.15.99:0/2298111498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 msgr2=0x55ee969f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 -- 172.21.15.99:0/2298111498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee966603f0 con 0x55ee969e9580 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f244fe00640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2455a00640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f9170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2448005fd0 tx=0x7f2448004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a43796bdeb6516 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.830+0000 7f2436a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448025070 con 0x55ee969e9580 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2448011d60 con 0x55ee969e9580 2024-03-23T23:46:38.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee96959800 con 0x55ee969e9580 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2448031050 con 0x55ee969e9580 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f245a627740 10 client.26171 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee9672a7d0 con 0x55ee969e9580 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.831+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ee96959800 con 0x55ee969e9580 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f244800f040 con 0x55ee969e9580 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 10 client.26171 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 10 client.26171 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 20 client.26171 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee969fd680 0x55ee969ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 4 client.26171 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee96953b50 con 0x55ee969fd680 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f2455000640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee969fd680 0x55ee969ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f2455000640 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee969fd680 0x55ee969ffac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c03f83fe12ea771c server_cookie=5c0cfbbc40939f02 in_seq=0 out_seq=0 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.832+0000 7f2436a00640 10 client.26171 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:38.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f2436a00640 1 -- 172.21.15.99:0/2298111498 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee96953b50 con 0x55ee969fd680 2024-03-23T23:46:38.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f2436a00640 10 client.26171 handle_command_reply: tid=0 2024-03-23T23:46:38.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 client.26171 shutdown 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 msgr2=0x55ee969f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f9170 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2448005fd0 tx=0x7f2448004a40 comp rx=0 tx=0).stop 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 shutdown_connections 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f9170 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e8bc0 0x55ee969f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee969fd680 0x55ee969ffac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 --2- 172.21.15.99:0/2298111498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e79f0 0x55ee969f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f2436a00640 0 client.26171 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.833+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 >> 172.21.15.99:0/2298111498 conn(0x55ee969e2de0 msgr2=0x55ee969e2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.834+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 shutdown_connections 2024-03-23T23:46:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.834+0000 7f245a627740 1 -- 172.21.15.99:0/2298111498 wait complete. 2024-03-23T23:46:38.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.834+0000 7f245a627740 20 client.26171 trim_cache size 0 max 16384 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.847+0000 7f245a627740 1 Processor -- start 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.847+0000 7f245a627740 1 -- start start 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.847+0000 7f245a627740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.847+0000 7f245a627740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 0x55ee969f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f245a627740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e9580 0x55ee969e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f245a627740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501aab20 con 0x55ee969e9580 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f245a627740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501ac4a0 con 0x55ee969e8bc0 2024-03-23T23:46:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f245a627740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501ab180 con 0x55ee969e79f0 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f244fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 0x55ee969f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57036/0 (socket says 172.21.15.99:57036) 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f244fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 0x55ee969f9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53040/0 (socket says 172.21.15.99:53040) 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 -- 172.21.15.99:0/2856898936 learned_addr learned my addr 172.21.15.99:0/2856898936 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455a00640 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e9580 0x55ee969e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 -- 172.21.15.99:0/2856898936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 msgr2=0x55ee969f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 0x55ee969f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 -- 172.21.15.99:0/2856898936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e9580 msgr2=0x55ee969e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e9580 0x55ee969e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 -- 172.21.15.99:0/2856898936 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee969e3830 con 0x55ee969e79f0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.848+0000 7f2455000640 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f244409b390 tx=0x7f24440d27b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95dac34c432efedf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f2441600640 1 -- 172.21.15.99:0/2856898936 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440b3070 con 0x55ee969e79f0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f2441600640 1 -- 172.21.15.99:0/2856898936 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f244409b690 con 0x55ee969e79f0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f2441600640 1 -- 172.21.15.99:0/2856898936 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244409aff0 con 0x55ee969e79f0 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 -- 172.21.15.99:0/2856898936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 msgr2=0x55ee969e18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f244409b390 tx=0x7f24440d27b0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 -- 172.21.15.99:0/2856898936 shutdown_connections 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e9580 0x55ee969e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e8bc0 0x55ee969f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 --2- 172.21.15.99:0/2856898936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969e18d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 -- 172.21.15.99:0/2856898936 >> 172.21.15.99:0/2856898936 conn(0x55ee969e2de0 msgr2=0x55ee969efae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 -- 172.21.15.99:0/2856898936 shutdown_connections 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.849+0000 7f245a627740 1 -- 172.21.15.99:0/2856898936 wait complete. 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 -- start start 2024-03-23T23:46:38.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 Processor -- start 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e8bc0 0x55ee969f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501abe40 con 0x55ee969e8bc0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501ab7e0 con 0x55ee969e9580 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24501a8ce0 con 0x55ee969e79f0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.850+0000 7f245a627740 10 client.0 _ll_register_callbacks cb 0x7f2448005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f244fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e8bc0 0x55ee969f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57044/0 (socket says 172.21.15.99:57044) 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 -- 172.21.15.99:0/3229207610 learned_addr learned my addr 172.21.15.99:0/3229207610 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455a00640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 -- 172.21.15.99:0/3229207610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 msgr2=0x55ee969f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 -- 172.21.15.99:0/3229207610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e8bc0 msgr2=0x55ee969f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e8bc0 0x55ee969f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 -- 172.21.15.99:0/3229207610 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee969f8050 con 0x55ee969e79f0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455a00640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2455000640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f244409b620 tx=0x7f24440d2820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c36b9ada8d614e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2436a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440b3030 con 0x55ee969e79f0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24440cc900 con 0x55ee969e79f0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.851+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f244409b690 con 0x55ee969e79f0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244409b1d0 con 0x55ee969e79f0 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 10 client.26212 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:38.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee969f9350 con 0x55ee969e79f0 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f244409b690 con 0x55ee969e79f0 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f244409b8f0 con 0x55ee969e79f0 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 10 client.26212 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 10 client.26212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 20 client.26212 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee96a59030 0x55ee96a5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 4 client.26212 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 --> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee968d1500 con 0x55ee96a59030 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.852+0000 7f244fe00640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee96a59030 0x55ee96a5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f244fe00640 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee96a59030 0x55ee96a5b420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9d176c7c657278c server_cookie=10e2aea8e0eaa05 in_seq=0 out_seq=0 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f2436a00640 10 client.26212 ms_handle_connect on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f2436a00640 1 -- 172.21.15.99:0/3229207610 <== mds.4 v2:172.21.15.174:6836/2363957261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee968d1500 con 0x55ee96a59030 2024-03-23T23:46:38.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f2436a00640 10 client.26212 handle_command_reply: tid=0 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 client.26212 shutdown 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 msgr2=0x55ee969f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f244409b620 tx=0x7f24440d2820 comp rx=0 tx=0).stop 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 shutdown_connections 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee969e9580 0x55ee969f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:6836/2363957261,v1:172.21.15.174:6837/2363957261] conn(0x55ee96a59030 0x55ee96a5b420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee969e8bc0 0x55ee969f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 --2- 172.21.15.99:0/3229207610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee969e79f0 0x55ee969f4500 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 >> 172.21.15.99:0/3229207610 conn(0x55ee969e2de0 msgr2=0x55ee969e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.853+0000 7f2436a00640 0 client.26212 ms_handle_reset on v2:172.21.15.174:6836/2363957261 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 shutdown_connections 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f245a627740 1 -- 172.21.15.99:0/3229207610 wait complete. 2024-03-23T23:46:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f245a627740 20 client.26212 trim_cache size 0 max 16384 2024-03-23T23:46:38.860 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 msgr2=0x7f24380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 0x7f24380a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ee969e8ac0 tx=0x7f2448009600 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 msgr2=0x7f2450075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9c0 tx=0x7f244400ee80 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 shutdown_connections 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f24380a2800 0x7f24380a4cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2450103150 0x7f2450073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2450102790 0x7f2450073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 --2- 172.21.15.99:0/943302652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501015a0 0x7f2450075450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.854+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 >> 172.21.15.99:0/943302652 conn(0x7f24500fcb10 msgr2=0x7f2450105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.855+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 shutdown_connections 2024-03-23T23:46:38.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:38.855+0000 7f2456400640 1 -- 172.21.15.99:0/943302652 wait complete. 2024-03-23T23:46:38.872 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:38.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-23T23:46:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 -- 172.21.15.99:0/1432701784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4100480 msgr2=0x7fb9c41008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 --2- 172.21.15.99:0/1432701784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4100480 0x7fb9c41008a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009900 tx=0x7fb9b802bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 -- 172.21.15.99:0/1432701784 shutdown_connections 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 --2- 172.21.15.99:0/1432701784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 0x7fb9c41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 --2- 172.21.15.99:0/1432701784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4101670 0x7fb9c4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 --2- 172.21.15.99:0/1432701784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4100480 0x7fb9c41008a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 -- 172.21.15.99:0/1432701784 >> 172.21.15.99:0/1432701784 conn(0x7fb9c40fba10 msgr2=0x7fb9c40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 -- 172.21.15.99:0/1432701784 shutdown_connections 2024-03-23T23:46:39.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.078+0000 7fb9cb200640 1 -- 172.21.15.99:0/1432701784 wait complete. 2024-03-23T23:46:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.080+0000 7fb9cb200640 1 Processor -- start 2024-03-23T23:46:39.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.080+0000 7fb9cb200640 1 -- start start 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4100480 0x7fb9c4069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 0x7fb9c419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c410c130 con 0x7fb9c4100480 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c410bfb0 con 0x7fb9c4101670 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9cb200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c410c2b0 con 0x7fb9c4102030 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4100480 0x7fb9c4069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53076/0 (socket says 172.21.15.99:53076) 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 -- 172.21.15.99:0/3796312411 learned_addr learned my addr 172.21.15.99:0/3796312411 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9ca800640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 0x7fb9c419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 -- 172.21.15.99:0/3796312411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 msgr2=0x7fb9c419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 0x7fb9c419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 -- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4100480 msgr2=0x7fb9c4069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.081+0000 7fb9c9400640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4100480 0x7fb9c4069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9c9400640 1 -- 172.21.15.99:0/3796312411 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c419f4d0 con 0x7fb9c4101670 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9c9400640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400e9f0 tx=0x7fb9b400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9bea00640 1 -- 172.21.15.99:0/3796312411 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400ccc0 con 0x7fb9c4101670 2024-03-23T23:46:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9bea00640 1 -- 172.21.15.99:0/3796312411 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4010910 con 0x7fb9c4101670 2024-03-23T23:46:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9bea00640 1 -- 172.21.15.99:0/3796312411 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40188a0 con 0x7fb9c4101670 2024-03-23T23:46:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c419f7c0 con 0x7fb9c4101670 2024-03-23T23:46:39.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.082+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c419fcc0 con 0x7fb9c4101670 2024-03-23T23:46:39.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.085+0000 7fb9bea00640 1 -- 172.21.15.99:0/3796312411 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb9b4010ac0 con 0x7fb9c4101670 2024-03-23T23:46:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.087+0000 7fb9cf5c9740 1 Processor -- start 2024-03-23T23:46:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.087+0000 7fb9cf5c9740 1 -- start start 2024-03-23T23:46:39.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.087+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e789f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f9580 0x559e789ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e7895c2c0 con 0x559e789f9580 2024-03-23T23:46:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e78904c90 con 0x559e789f79f0 2024-03-23T23:46:39.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e78958360 con 0x559e789f8bc0 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9bea00640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 0x7fb9ac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e789f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57082/0 (socket says 172.21.15.99:57082) 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 -- 172.21.15.99:0/2041810 learned_addr learned my addr 172.21.15.99:0/2041810 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9400640 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f9580 0x559e789ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 -- 172.21.15.99:0/2041810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 msgr2=0x559e789f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e789f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9bea00640 1 -- 172.21.15.99:0/3796312411 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9b40cee80 con 0x7fb9c4101670 2024-03-23T23:46:39.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 -- 172.21.15.99:0/2041810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f9580 msgr2=0x559e789ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f9580 0x559e789ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9c9e00640 1 -- 172.21.15.99:0/2041810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e78929080 con 0x559e789f8bc0 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9ca800640 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e789f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.088+0000 7fb9ca800640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 0x7fb9ac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.089+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243cfc322eaf24ec server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.089+0000 7fb9b3000640 1 -- 172.21.15.99:0/2041810 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b802c980 con 0x559e789f8bc0 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.089+0000 7fb9b3000640 1 -- 172.21.15.99:0/2041810 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b802cb20 con 0x559e789f8bc0 2024-03-23T23:46:39.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.089+0000 7fb9b3000640 1 -- 172.21.15.99:0/2041810 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80046c0 con 0x559e789f8bc0 2024-03-23T23:46:39.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/2041810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 msgr2=0x559e789f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8009560 comp rx=0 tx=0).stop 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9ca800640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 0x7fb9ac0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00049d0 tx=0x7fb9c0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/2041810 shutdown_connections 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f9580 0x559e789ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f8bc0 0x559e789f9040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/2041810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e789f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/2041810 >> 172.21.15.99:0/2041810 conn(0x559e789f2de0 msgr2=0x559e789f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/2041810 shutdown_connections 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/2041810 wait complete. 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.090+0000 7fb9cf5c9740 1 -- start start 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 Processor -- start 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a05ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e78a04940 con 0x559e789f8bc0 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e789180d0 con 0x559e789f79f0 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e78a04ac0 con 0x559e789f9580 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9cf5c9740 10 client.0 _ll_register_callbacks cb 0x559e7891ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57148/0 (socket says 172.21.15.99:57148) 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57086/0 (socket says 172.21.15.99:57086) 2024-03-23T23:46:39.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.091+0000 7fb9c9400640 1 -- 172.21.15.99:0/4228994684 learned_addr learned my addr 172.21.15.99:0/4228994684 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a05ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 -- 172.21.15.99:0/4228994684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 msgr2=0x559e78a05ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a05ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 -- 172.21.15.99:0/4228994684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 msgr2=0x559e78a06530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 -- 172.21.15.99:0/4228994684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e786703f0 con 0x559e789f9580 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9c9400640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9ca800640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x559e789f8ac0 tx=0x7fb9c0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce954b4d1cb38fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9a7400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0010640 con 0x559e789f9580 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b802cb20 con 0x559e789f9580 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.092+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9c0018050 con 0x559e789f9580 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c00107e0 con 0x559e789f9580 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 10 client.26224 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e7873a7d0 con 0x559e789f9580 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b802cb20 con 0x559e789f9580 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9c001a070 con 0x559e789f9580 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 10 client.26224 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 10 client.26224 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 20 client.26224 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e78a11a60 0x559e78a13e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 4 client.26224 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.093+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e78969800 con 0x559e78a11a60 2024-03-23T23:46:39.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.094+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e78a11a60 0x559e78a13e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.094+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e78a11a60 0x559e78a13e80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41d3c92767286464 server_cookie=689f9346667f5971 in_seq=0 out_seq=0 2024-03-23T23:46:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.094+0000 7fb9a7400640 10 client.26224 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9a7400640 1 -- 172.21.15.99:0/4228994684 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e78969800 con 0x559e78a11a60 2024-03-23T23:46:39.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9a7400640 10 client.26224 handle_command_reply: tid=0 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 client.26224 shutdown 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 msgr2=0x559e78a0d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x559e789f8ac0 tx=0x7fb9c0009500 comp rx=0 tx=0).stop 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 shutdown_connections 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e78a11a60 0x559e78a13e80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f9580 0x559e78a0d570 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a06530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/4228994684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a05ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9a7400640 0 client.26224 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 >> 172.21.15.99:0/4228994684 conn(0x559e789f2de0 msgr2=0x559e789f2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 shutdown_connections 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.095+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/4228994684 wait complete. 2024-03-23T23:46:39.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.096+0000 7fb9cf5c9740 20 client.26224 trim_cache size 0 max 16384 2024-03-23T23:46:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 Processor -- start 2024-03-23T23:46:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 -- start start 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 0x559e789f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a6680 con 0x559e789f8bc0 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a8000 con 0x559e789f2de0 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a6ce0 con 0x559e789f79f0 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.109+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 0x559e789f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57098/0 (socket says 172.21.15.99:57098) 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9400640 1 -- 172.21.15.99:0/1594466753 learned_addr learned my addr 172.21.15.99:0/1594466753 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 -- 172.21.15.99:0/1594466753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 msgr2=0x559e78a0d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 -- 172.21.15.99:0/1594466753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 msgr2=0x559e78a18740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a18740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 -- 172.21.15.99:0/1594466753 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e78a00140 con 0x559e789f2de0 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9400640 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 0x559e789f1fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9b802c7b0 tx=0x7fb9b8041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6911cd7e7b89d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9b2600640 1 -- 172.21.15.99:0/1594466753 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8050070 con 0x559e789f2de0 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9b2600640 1 -- 172.21.15.99:0/1594466753 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b802c980 con 0x559e789f2de0 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.110+0000 7fb9b2600640 1 -- 172.21.15.99:0/1594466753 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b803f6a0 con 0x559e789f2de0 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/1594466753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 msgr2=0x559e789f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 0x559e789f1fe0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9b802c7b0 tx=0x7fb9b8041040 comp rx=0 tx=0).stop 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/1594466753 shutdown_connections 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a18740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f79f0 0x559e78a0d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/1594466753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f2de0 0x559e789f1fe0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/1594466753 >> 172.21.15.99:0/1594466753 conn(0x559e78a05200 msgr2=0x559e789f18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/1594466753 shutdown_connections 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.111+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/1594466753 wait complete. 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 -- start start 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 Processor -- start 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 0x559e78a09530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a79a0 con 0x559e789f8bc0 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a7340 con 0x559e789f79f0 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41a4840 con 0x559e789f2de0 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9cf5c9740 10 client.0 _ll_register_callbacks cb 0x7fb9c0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 0x559e78a09530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 0x559e78a09530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57100/0 (socket says 172.21.15.99:57100) 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53122/0 (socket says 172.21.15.99:53122) 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9c9e00640 1 -- 172.21.15.99:0/162837481 learned_addr learned my addr 172.21.15.99:0/162837481 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.112+0000 7fb9ca800640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 -- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 msgr2=0x559e78a09530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 0x559e78a09530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 -- 172.21.15.99:0/162837481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 msgr2=0x559e78a0ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a0ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 -- 172.21.15.99:0/162837481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e78a0d080 con 0x559e789f79f0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9c9400640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb9b409b730 tx=0x7fb9b4009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e6f11f38283c7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9a7400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40a3f40 con 0x559e789f79f0 2024-03-23T23:46:39.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b802c980 con 0x559e789f79f0 2024-03-23T23:46:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb9b40860b0 con 0x559e789f79f0 2024-03-23T23:46:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.113+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9b40afb80 con 0x559e789f79f0 2024-03-23T23:46:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 10 client.26195 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e78a6dc60 con 0x559e789f79f0 2024-03-23T23:46:39.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9b802c980 con 0x559e789f79f0 2024-03-23T23:46:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fb9b409cad0 con 0x559e789f79f0 2024-03-23T23:46:39.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 10 client.26195 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 10 client.26195 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.i 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 20 client.26195 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e789f9580 0x559e78a705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 4 client.26195 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 --> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e78963b50 con 0x559e789f9580 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.114+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e789f9580 0x559e78a705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9a7400640 10 client.26195 ms_handle_connect on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9c9e00640 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e789f9580 0x559e78a705f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cda0433af2be0377 server_cookie=9e75b4bf8986ed96 in_seq=0 out_seq=0 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9a7400640 1 -- 172.21.15.99:0/162837481 <== mds.3 v2:172.21.15.174:6838/1404645838 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e78963b50 con 0x559e789f9580 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9a7400640 10 client.26195 handle_command_reply: tid=0 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 client.26195 shutdown 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 msgr2=0x559e78a09a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb9b409b730 tx=0x7fb9b4009700 comp rx=0 tx=0).stop 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 shutdown_connections 2024-03-23T23:46:39.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:6838/1404645838,v1:172.21.15.174:6839/1404645838] conn(0x559e789f9580 0x559e78a705f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e789f8bc0 0x559e78a0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e789f79f0 0x559e78a09a70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 --2- 172.21.15.99:0/162837481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e789f2de0 0x559e78a09530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9a7400640 0 client.26195 ms_handle_reset on v2:172.21.15.174:6838/1404645838 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.115+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 >> 172.21.15.99:0/162837481 conn(0x559e78a05200 msgr2=0x559e789f2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 shutdown_connections 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cf5c9740 1 -- 172.21.15.99:0/162837481 wait complete. 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cf5c9740 20 client.26195 trim_cache size 0 max 16384 2024-03-23T23:46:39.120 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 msgr2=0x7fb9ac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 0x7fb9ac0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00049d0 tx=0x7fb9c0004a40 comp rx=0 tx=0).stop 2024-03-23T23:46:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 msgr2=0x7fb9c419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.116+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400e9f0 tx=0x7fb9b400edf0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 shutdown_connections 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb9ac0a2880 0x7fb9ac0a4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4102030 0x7fb9c419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4101670 0x7fb9c419c5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 --2- 172.21.15.99:0/3796312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9c4100480 0x7fb9c4069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 >> 172.21.15.99:0/3796312411 conn(0x7fb9c40fba10 msgr2=0x7fb9c4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 shutdown_connections 2024-03-23T23:46:39.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.117+0000 7fb9cb200640 1 -- 172.21.15.99:0/3796312411 wait complete. 2024-03-23T23:46:39.132 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:38 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2856270061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:39.135 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:39.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T23:46:39.172 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:38 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2856270061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:39.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:38 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2856270061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:46:39.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 -- 172.21.15.99:0/3537810347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0104a60 msgr2=0x7f47b0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 --2- 172.21.15.99:0/3537810347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0104a60 0x7f47b0106e50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f479c009900 tx=0x7f479c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 -- 172.21.15.99:0/3537810347 shutdown_connections 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 --2- 172.21.15.99:0/3537810347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0107390 0x7f47b01097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 --2- 172.21.15.99:0/3537810347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0104a60 0x7f47b0106e50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 --2- 172.21.15.99:0/3537810347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 -- 172.21.15.99:0/3537810347 >> 172.21.15.99:0/3537810347 conn(0x7f47b00fbb40 msgr2=0x7f47b00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 -- 172.21.15.99:0/3537810347 shutdown_connections 2024-03-23T23:46:39.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.343+0000 7f47b8200640 1 -- 172.21.15.99:0/3537810347 wait complete. 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.344+0000 7f47b8200640 1 Processor -- start 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.344+0000 7f47b8200640 1 -- start start 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.344+0000 7f47b8200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.344+0000 7f47b8200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0104a60 0x7f47b01000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b8200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0107390 0x7f47b0100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b8200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b010d370 con 0x7f47b0104a60 2024-03-23T23:46:39.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b8200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b010d1f0 con 0x7f47b0107390 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b8200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b010d4f0 con 0x7f47b0102130 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57124/0 (socket says 172.21.15.99:57124) 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 -- 172.21.15.99:0/3433067561 learned_addr learned my addr 172.21.15.99:0/3433067561 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b4e00640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0104a60 0x7f47b01000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b6200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0107390 0x7f47b0100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 -- 172.21.15.99:0/3433067561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0107390 msgr2=0x7f47b0100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0107390 0x7f47b0100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 -- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0104a60 msgr2=0x7f47b01000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0104a60 0x7f47b01000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.345+0000 7f47b5800640 1 -- 172.21.15.99:0/3433067561 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0100eb0 con 0x7f47b0102130 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47b5800640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f47a400e9f0 tx=0x7f47a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47ae000640 1 -- 172.21.15.99:0/3433067561 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400cd80 con 0x7f47b0102130 2024-03-23T23:46:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47ae000640 1 -- 172.21.15.99:0/3433067561 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47a4010910 con 0x7f47b0102130 2024-03-23T23:46:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47ae000640 1 -- 172.21.15.99:0/3433067561 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4018990 con 0x7f47b0102130 2024-03-23T23:46:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b01a1b40 con 0x7f47b0102130 2024-03-23T23:46:39.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.346+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b01a20b0 con 0x7f47b0102130 2024-03-23T23:46:39.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.349+0000 7f47ae000640 1 -- 172.21.15.99:0/3433067561 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f47a4010ac0 con 0x7f47b0102130 2024-03-23T23:46:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.350+0000 7f47bafb3740 1 Processor -- start 2024-03-23T23:46:39.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.350+0000 7f47bafb3740 1 -- start start 2024-03-23T23:46:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccaec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccb6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46cc132c0 con 0x55e46ccae810 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46cbbbc90 con 0x55e46ccaf9e0 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47bafb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46cc0f360 con 0x55e46ccb03a0 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47ae000640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 0x7f47980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47b4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccb6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53146/0 (socket says 172.21.15.99:53146) 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47b6200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccaec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.351+0000 7f47b5800640 1 -- 172.21.15.99:0/2627046408 learned_addr learned my addr 172.21.15.99:0/2627046408 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47ae000640 1 -- 172.21.15.99:0/3433067561 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47a40ceda0 con 0x7f47b0102130 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 -- 172.21.15.99:0/2627046408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 msgr2=0x55e46ccb6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccb6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 -- 172.21.15.99:0/2627046408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 msgr2=0x55e46ccaec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccaec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 -- 172.21.15.99:0/2627046408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46cbe0080 con 0x55e46ccaf9e0 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b4e00640 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccb6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b5800640 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f47a4097bd0 tx=0x7f47a4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e2f31f18089a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47b6200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 0x7f47980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47a2a00640 1 -- 172.21.15.99:0/2627046408 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40a3e60 con 0x55e46ccaf9e0 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47a2a00640 1 -- 172.21.15.99:0/2627046408 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47a40cc930 con 0x55e46ccaf9e0 2024-03-23T23:46:39.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.352+0000 7f47a2a00640 1 -- 172.21.15.99:0/2627046408 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a409abc0 con 0x55e46ccaf9e0 2024-03-23T23:46:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 -- 172.21.15.99:0/2627046408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 msgr2=0x55e46ccafe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47b6200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 0x7f47980a4d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e46ccaf8e0 tx=0x7f47a8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f47a4097bd0 tx=0x7f47a4018420 comp rx=0 tx=0).stop 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 -- 172.21.15.99:0/2627046408 shutdown_connections 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccb6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccafe60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 --2- 172.21.15.99:0/2627046408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccaec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 -- 172.21.15.99:0/2627046408 >> 172.21.15.99:0/2627046408 conn(0x55e46cca9c00 msgr2=0x55e46ccaa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 -- 172.21.15.99:0/2627046408 shutdown_connections 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.353+0000 7f47bafb3740 1 -- 172.21.15.99:0/2627046408 wait complete. 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.354+0000 7f47bafb3740 1 -- start start 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.354+0000 7f47bafb3740 1 Processor -- start 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 0x55e46ccb8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ccb7350 con 0x55e46ccaf9e0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46cbcf0d0 con 0x55e46ccae810 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46ccb74d0 con 0x55e46ccb03a0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47bafb3740 10 client.0 _ll_register_callbacks cb 0x55e46cbd3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b6200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53150/0 (socket says 172.21.15.99:53150) 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 -- 172.21.15.99:0/4208359732 learned_addr learned my addr 172.21.15.99:0/4208359732 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b4e00640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 0x55e46ccb8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 -- 172.21.15.99:0/4208359732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 msgr2=0x55e46ccbffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccbffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 -- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 msgr2=0x55e46ccb8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 0x55e46ccb8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b5800640 1 -- 172.21.15.99:0/4208359732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e46c9273f0 con 0x55e46ccae810 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b4e00640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 0x55e46ccb8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:39.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.355+0000 7f47b6200640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47b5800640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f47a40a3930 tx=0x7f47a4018e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9ab7f5c6dd8415 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40b3020 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47a409add0 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47a40cc930 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a40a3e60 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47bafb3740 10 client.26207 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e46c9f17d0 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47a40cc930 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f47a40aa020 con 0x55e46ccae810 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47bafb3740 10 client.26207 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.356+0000 7f47bafb3740 10 client.26207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.357+0000 7f47bafb3740 20 client.26207 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.357+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46ccc4100 0x55e46ccc64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.357+0000 7f47bafb3740 4 client.26207 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.357+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e46cc20800 con 0x55e46ccc4100 2024-03-23T23:46:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.357+0000 7f47b4e00640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46ccc4100 0x55e46ccc64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.358+0000 7f47b4e00640 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46ccc4100 0x55e46ccc64f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33f3980cab3a5a0 server_cookie=c7f3e04c98495934 in_seq=0 out_seq=0 2024-03-23T23:46:39.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.358+0000 7f4797400640 10 client.26207 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.358+0000 7f4797400640 1 -- 172.21.15.99:0/4208359732 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e46cc20800 con 0x55e46ccc4100 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.358+0000 7f4797400640 10 client.26207 handle_command_reply: tid=0 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.358+0000 7f47bafb3740 1 client.26207 shutdown 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 msgr2=0x55e46ccb8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f47a40a3930 tx=0x7f47a4018e10 comp rx=0 tx=0).stop 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 shutdown_connections 2024-03-23T23:46:39.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46ccc4100 0x55e46ccc64f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccaf9e0 0x55e46ccb8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f4797400640 0 client.26207 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 --2- 172.21.15.99:0/4208359732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccb8a20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 >> 172.21.15.99:0/4208359732 conn(0x55e46cca9c00 msgr2=0x55e46cca8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 shutdown_connections 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 1 -- 172.21.15.99:0/4208359732 wait complete. 2024-03-23T23:46:39.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.359+0000 7f47bafb3740 20 client.26207 trim_cache size 0 max 16384 2024-03-23T23:46:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.378+0000 7f47bafb3740 1 Processor -- start 2024-03-23T23:46:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 -- start start 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccb6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01aac90 con 0x55e46ccae810 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01ac610 con 0x55e46ccaf9e0 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47bafb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01ab2f0 con 0x55e46ccb03a0 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47b4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47b6200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47b4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53160/0 (socket says 172.21.15.99:53160) 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47b4e00640 1 -- 172.21.15.99:0/1882019994 learned_addr learned my addr 172.21.15.99:0/1882019994 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.379+0000 7f47b5800640 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccb6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 -- 172.21.15.99:0/1882019994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 msgr2=0x55e46ccad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 -- 172.21.15.99:0/1882019994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 msgr2=0x55e46ccb6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccb6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 -- 172.21.15.99:0/1882019994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e46ccb8630 con 0x55e46ccaf9e0 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47b4e00640 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47b006a7d0 tx=0x7f479c02df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9618c7db11713bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47a2000640 1 -- 172.21.15.99:0/1882019994 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c04e020 con 0x55e46ccaf9e0 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47a2000640 1 -- 172.21.15.99:0/1882019994 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f479c02cbc0 con 0x55e46ccaf9e0 2024-03-23T23:46:39.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.380+0000 7f47a2000640 1 -- 172.21.15.99:0/1882019994 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c048070 con 0x55e46ccaf9e0 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 -- 172.21.15.99:0/1882019994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 msgr2=0x55e46ccbff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f47b006a7d0 tx=0x7f479c02df90 comp rx=0 tx=0).stop 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 -- 172.21.15.99:0/1882019994 shutdown_connections 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccb03a0 0x55e46ccad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbff60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 --2- 172.21.15.99:0/1882019994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccae810 0x55e46ccb6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 -- 172.21.15.99:0/1882019994 >> 172.21.15.99:0/1882019994 conn(0x55e46cca9c00 msgr2=0x55e46ccace60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 -- 172.21.15.99:0/1882019994 shutdown_connections 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.381+0000 7f47bafb3740 1 -- 172.21.15.99:0/1882019994 wait complete. 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.382+0000 7f47bafb3740 1 -- start start 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.382+0000 7f47bafb3740 1 Processor -- start 2024-03-23T23:46:39.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.382+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.382+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47bafb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 0x55e46ccbedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47bafb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01abfb0 con 0x55e46ccb03a0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47bafb3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01ab950 con 0x55e46ccae810 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47bafb3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01a8e50 con 0x55e46ccaf9e0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47bafb3740 10 client.0 _ll_register_callbacks cb 0x7f47a8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53174/0 (socket says 172.21.15.99:53174) 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 -- 172.21.15.99:0/880967736 learned_addr learned my addr 172.21.15.99:0/880967736 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57172/0 (socket says 172.21.15.99:57172) 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b6200640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 0x55e46ccbedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 -- 172.21.15.99:0/880967736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 msgr2=0x55e46ccbbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 -- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 msgr2=0x55e46ccbedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 0x55e46ccbedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 -- 172.21.15.99:0/880967736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e46ccbf3e0 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b6200640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 0x55e46ccbedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f47b5800640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f47a4097910 tx=0x7f47a400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e88527af5b575c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.383+0000 7f4797400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40b3040 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47a40b4220 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f479c02cbc0 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a4097f30 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 10 client.26213 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e46cd1fa60 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f479c02cbc0 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f47a40aa040 con 0x55e46ccae810 2024-03-23T23:46:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 10 client.26213 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 10 client.26213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.a 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 20 client.26213 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46cd20100 0x55e46cd224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 4 client.26213 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.384+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 --> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e46cc1ab50 con 0x55e46cd20100 2024-03-23T23:46:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.385+0000 7f47b4e00640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46cd20100 0x55e46cd224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.385+0000 7f47b4e00640 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46cd20100 0x55e46cd224f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a8adb98fa838493 server_cookie=df52e50df92fd59c in_seq=0 out_seq=0 2024-03-23T23:46:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.385+0000 7f4797400640 10 client.26213 ms_handle_connect on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f4797400640 1 -- 172.21.15.99:0/880967736 <== mds.2 v2:172.21.15.99:6840/678226276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e46cc1ab50 con 0x55e46cd20100 2024-03-23T23:46:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f4797400640 10 client.26213 handle_command_reply: tid=0 2024-03-23T23:46:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 client.26213 shutdown 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 msgr2=0x55e46ccbb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f47a4097910 tx=0x7f47a400cf90 comp rx=0 tx=0).stop 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 shutdown_connections 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:6840/678226276,v1:172.21.15.99:6841/678226276] conn(0x55e46cd20100 0x55e46cd224f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e46ccb03a0 0x55e46ccbedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e46ccaf9e0 0x55e46ccbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f4797400640 0 client.26213 ms_handle_reset on v2:172.21.15.99:6840/678226276 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 --2- 172.21.15.99:0/880967736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e46ccae810 0x55e46ccbb890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 >> 172.21.15.99:0/880967736 conn(0x55e46cca9c00 msgr2=0x55e46ccad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 shutdown_connections 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 1 -- 172.21.15.99:0/880967736 wait complete. 2024-03-23T23:46:39.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.386+0000 7f47bafb3740 20 client.26213 trim_cache size 0 max 16384 2024-03-23T23:46:39.391 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 msgr2=0x7f47980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 0x7f47980a4d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55e46ccaf8e0 tx=0x7f47a8004a40 comp rx=0 tx=0).stop 2024-03-23T23:46:39.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 msgr2=0x7f47b0101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f47a400e9f0 tx=0x7f47a400eeb0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 shutdown_connections 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f47980a2880 0x7f47980a4d40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b0107390 0x7f47b0100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47b0104a60 0x7f47b01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 --2- 172.21.15.99:0/3433067561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102130 0x7f47b0101aa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 >> 172.21.15.99:0/3433067561 conn(0x7f47b00fbb40 msgr2=0x7f47b0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 shutdown_connections 2024-03-23T23:46:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.387+0000 7f47b8200640 1 -- 172.21.15.99:0/3433067561 wait complete. 2024-03-23T23:46:39.411 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:39.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T23:46:39.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 -- 172.21.15.99:0/2885431904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 msgr2=0x7f316c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 --2- 172.21.15.99:0/2885431904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c074580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3158009900 tx=0x7f315802bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 -- 172.21.15.99:0/2885431904 shutdown_connections 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 --2- 172.21.15.99:0/2885431904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c109460 0x7f316c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 --2- 172.21.15.99:0/2885431904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c074580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 --2- 172.21.15.99:0/2885431904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c0756e0 0x7f316c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 -- 172.21.15.99:0/2885431904 >> 172.21.15.99:0/2885431904 conn(0x7f316c0fcb10 msgr2=0x7f316c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 -- 172.21.15.99:0/2885431904 shutdown_connections 2024-03-23T23:46:39.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.603+0000 7f3171a00640 1 -- 172.21.15.99:0/2885431904 wait complete. 2024-03-23T23:46:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 Processor -- start 2024-03-23T23:46:39.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 -- start start 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 0x7f316c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 0x7f316c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10f450 con 0x7f316c109460 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10f2d0 con 0x7f316c0756e0 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.605+0000 7f3171a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10f5d0 con 0x7f316c074100 2024-03-23T23:46:39.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 0x7f316c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57190/0 (socket says 172.21.15.99:57190) 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316be00640 1 -- 172.21.15.99:0/487333227 learned_addr learned my addr 172.21.15.99:0/487333227 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f3171000640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 0x7f316c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 -- 172.21.15.99:0/487333227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 msgr2=0x7f316c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 -- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 msgr2=0x7f316c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 0x7f316c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 -- 172.21.15.99:0/487333227 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c1a16a0 con 0x7f316c0756e0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316be00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f3171000640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 0x7f316c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.606+0000 7f316b400640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 0x7f316c19a010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3158036040 tx=0x7f315802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.607+0000 7f3168c00640 1 -- 172.21.15.99:0/487333227 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158043020 con 0x7f316c0756e0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.607+0000 7f3168c00640 1 -- 172.21.15.99:0/487333227 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31580054e0 con 0x7f316c0756e0 2024-03-23T23:46:39.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.607+0000 7f3168c00640 1 -- 172.21.15.99:0/487333227 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158003cb0 con 0x7f316c0756e0 2024-03-23T23:46:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.607+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c1a1930 con 0x7f316c0756e0 2024-03-23T23:46:39.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.607+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c1a1ea0 con 0x7f316c0756e0 2024-03-23T23:46:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.610+0000 7f3168c00640 1 -- 172.21.15.99:0/487333227 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3158046020 con 0x7f316c0756e0 2024-03-23T23:46:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 Processor -- start 2024-03-23T23:46:39.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 -- start start 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x55950329b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 0x5595032a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595031fe2c0 con 0x55950329abc0 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595031a6c90 con 0x55950329b580 2024-03-23T23:46:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.611+0000 7f3175da5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595031fa360 con 0x5595032999f0 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57200/0 (socket says 172.21.15.99:57200) 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 -- 172.21.15.99:0/1644715458 learned_addr learned my addr 172.21.15.99:0/1644715458 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f316b400640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 0x5595032a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f316be00640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x55950329b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3168c00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 0x7f31500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3168c00640 1 -- 172.21.15.99:0/487333227 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f31580f0b50 con 0x7f316c0756e0 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 -- 172.21.15.99:0/1644715458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 msgr2=0x5595032a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 0x5595032a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 -- 172.21.15.99:0/1644715458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 msgr2=0x55950329b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x55950329b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 -- 172.21.15.99:0/1644715458 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595031cb080 con 0x5595032999f0 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f316b400640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 0x5595032a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.612+0000 7f3171000640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 0x7f31500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.613+0000 7f3171000640 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3164009fd0 tx=0x7f3164010a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5284d7bed4ce7574 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.613+0000 7f3161600640 1 -- 172.21.15.99:0/1644715458 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164019070 con 0x5595032999f0 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.613+0000 7f3161600640 1 -- 172.21.15.99:0/1644715458 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3164022070 con 0x5595032999f0 2024-03-23T23:46:39.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.613+0000 7f3161600640 1 -- 172.21.15.99:0/1644715458 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164026c40 con 0x5595032999f0 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3171000640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 0x7f31500a4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3164021000 tx=0x7f316401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- 172.21.15.99:0/1644715458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 msgr2=0x559503299e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3164009fd0 tx=0x7f3164010a00 comp rx=0 tx=0).stop 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- 172.21.15.99:0/1644715458 shutdown_connections 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55950329b580 0x5595032a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x55950329b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 --2- 172.21.15.99:0/1644715458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595032999f0 0x559503299e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- 172.21.15.99:0/1644715458 >> 172.21.15.99:0/1644715458 conn(0x559503294de0 msgr2=0x5595032951e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- 172.21.15.99:0/1644715458 shutdown_connections 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- 172.21.15.99:0/1644715458 wait complete. 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.614+0000 7f3175da5740 1 -- start start 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 Processor -- start 2024-03-23T23:46:39.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 0x5595032a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329b580 0x5595032ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595032a2590 con 0x55950329b580 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595031ba0d0 con 0x5595032999f0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595032a2710 con 0x55950329abc0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3175da5740 10 client.0 _ll_register_callbacks cb 0x5595031bee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53216/0 (socket says 172.21.15.99:53216) 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f316be00640 1 -- 172.21.15.99:0/2158625691 learned_addr learned my addr 172.21.15.99:0/2158625691 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f316b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 0x5595032a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.615+0000 7f3171000640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329b580 0x5595032ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 -- 172.21.15.99:0/2158625691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 msgr2=0x5595032a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 0x5595032a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 -- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329b580 msgr2=0x5595032ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329b580 0x5595032ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 -- 172.21.15.99:0/2158625691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559502f123f0 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316b400640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 0x5595032a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f316be00640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f315c011990 tx=0x7f315c011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9cf0c134941fb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f3156000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c010c40 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f315c010de0 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.616+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3164022070 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315c021070 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 10 client.26225 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559502fdc7d0 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3164022070 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f315c01b420 con 0x5595032999f0 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 10 client.26225 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 10 client.26225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 20 client.26225 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5595032af730 0x5595032b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 4 client.26225 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.617+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55950320b800 con 0x5595032af730 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.618+0000 7f316b400640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5595032af730 0x5595032b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.618+0000 7f3156000640 10 client.26225 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:39.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.618+0000 7f316b400640 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5595032af730 0x5595032b1bd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9905983098813806 server_cookie=88d9ec9913130725 in_seq=0 out_seq=0 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.619+0000 7f3156000640 1 -- 172.21.15.99:0/2158625691 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55950320b800 con 0x5595032af730 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.619+0000 7f3156000640 10 client.26225 handle_command_reply: tid=0 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.619+0000 7f3175da5740 1 client.26225 shutdown 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 msgr2=0x5595032a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f315c011990 tx=0x7f315c011e50 comp rx=0 tx=0).stop 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 shutdown_connections 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x5595032af730 0x5595032b1bd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329b580 0x5595032ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329abc0 0x5595032a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 --2- 172.21.15.99:0/2158625691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032a3c30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3156000640 0 client.26225 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 >> 172.21.15.99:0/2158625691 conn(0x559503294de0 msgr2=0x559503294160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 shutdown_connections 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 1 -- 172.21.15.99:0/2158625691 wait complete. 2024-03-23T23:46:39.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.620+0000 7f3175da5740 20 client.26225 trim_cache size 0 max 16384 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 Processor -- start 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 -- start start 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1aa9c0 con 0x55950329abc0 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1ac340 con 0x5595032999f0 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f3175da5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1ab020 con 0x55950329b580 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.634+0000 7f316b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53232/0 (socket says 172.21.15.99:53232) 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 -- 172.21.15.99:0/4156390952 learned_addr learned my addr 172.21.15.99:0/4156390952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f3171000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 -- 172.21.15.99:0/4156390952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 msgr2=0x5595032b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 -- 172.21.15.99:0/4156390952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 msgr2=0x5595032ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 -- 172.21.15.99:0/4156390952 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595032908b0 con 0x5595032999f0 2024-03-23T23:46:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f316be00640 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f315c01b810 tx=0x7f315c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8795f342ecd5f0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f3160c00640 1 -- 172.21.15.99:0/4156390952 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c01b420 con 0x5595032999f0 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f3160c00640 1 -- 172.21.15.99:0/4156390952 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f315c01b5c0 con 0x5595032999f0 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.635+0000 7f3160c00640 1 -- 172.21.15.99:0/4156390952 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0054f0 con 0x5595032999f0 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 -- 172.21.15.99:0/4156390952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 msgr2=0x559503298040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f315c01b810 tx=0x7f315c0035f0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 -- 172.21.15.99:0/4156390952 shutdown_connections 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 --2- 172.21.15.99:0/4156390952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x559503298040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 -- 172.21.15.99:0/4156390952 >> 172.21.15.99:0/4156390952 conn(0x559503294de0 msgr2=0x559503294160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 -- 172.21.15.99:0/4156390952 shutdown_connections 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.636+0000 7f3175da5740 1 -- 172.21.15.99:0/4156390952 wait complete. 2024-03-23T23:46:39.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 -- start start 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 Processor -- start 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1abce0 con 0x55950329abc0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1ab680 con 0x5595032999f0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1a8b80 con 0x55950329b580 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f3175da5740 10 client.0 _ll_register_callbacks cb 0x7f315c00be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f316b400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f316be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53234/0 (socket says 172.21.15.99:53234) 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.637+0000 7f316be00640 1 -- 172.21.15.99:0/403007357 learned_addr learned my addr 172.21.15.99:0/403007357 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f3171000640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 -- 172.21.15.99:0/403007357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 msgr2=0x5595032a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 -- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 msgr2=0x5595032ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 -- 172.21.15.99:0/403007357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595032a99f0 con 0x5595032999f0 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f316be00640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f315c01b890 tx=0x7f315c011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36137001941ab30 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f3156000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c010e50 con 0x5595032999f0 2024-03-23T23:46:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f315c017040 con 0x5595032999f0 2024-03-23T23:46:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.638+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f315c01b5c0 con 0x5595032999f0 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315c0053f0 con 0x5595032999f0 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 10 client.26231 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55950330bb40 con 0x5595032999f0 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f315c01b5c0 con 0x5595032999f0 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f315c026480 con 0x5595032999f0 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 10 client.26231 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 10 client.26231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 20 client.26231 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55950330c540 0x55950330e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 4 client.26231 mds_command: new command op to 24565 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 --> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559503205b50 con 0x55950330c540 2024-03-23T23:46:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.639+0000 7f316b400640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55950330c540 0x55950330e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f316b400640 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55950330c540 0x55950330e960 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70d866835b04a882 server_cookie=66fa0878b69dce34 in_seq=0 out_seq=0 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3156000640 10 client.26231 ms_handle_connect on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3156000640 1 -- 172.21.15.99:0/403007357 <== mds.1 v2:172.21.15.99:6838/4063961606 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559503205b50 con 0x55950330c540 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3156000640 10 client.26231 handle_command_reply: tid=0 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3175da5740 1 client.26231 shutdown 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 msgr2=0x5595032aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.640+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f315c01b890 tx=0x7f315c011e50 comp rx=0 tx=0).stop 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 shutdown_connections 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:6838/4063961606,v1:172.21.15.99:6839/4063961606] conn(0x55950330c540 0x55950330e960 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55950329b580 0x5595032a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55950329abc0 0x5595032ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 --2- 172.21.15.99:0/403007357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5595032999f0 0x5595032aabb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3156000640 0 client.26231 ms_handle_reset on v2:172.21.15.99:6838/4063961606 2024-03-23T23:46:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 >> 172.21.15.99:0/403007357 conn(0x559503294de0 msgr2=0x55950329ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 shutdown_connections 2024-03-23T23:46:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 1 -- 172.21.15.99:0/403007357 wait complete. 2024-03-23T23:46:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3175da5740 20 client.26231 trim_cache size 0 max 16384 2024-03-23T23:46:39.648 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 msgr2=0x7f31500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 0x7f31500a4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3164021000 tx=0x7f316401f000 comp rx=0 tx=0).stop 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 msgr2=0x7f316c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.641+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 0x7f316c19a010 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3158036040 tx=0x7f315802c6c0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 shutdown_connections 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f31500a2880 0x7f31500a4d40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f316c109460 0x7f316c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f316c0756e0 0x7f316c19a010 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 --2- 172.21.15.99:0/487333227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f316c074100 0x7f316c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 >> 172.21.15.99:0/487333227 conn(0x7f316c0fcb10 msgr2=0x7f316c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 shutdown_connections 2024-03-23T23:46:39.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.642+0000 7f3171a00640 1 -- 172.21.15.99:0/487333227 wait complete. 2024-03-23T23:46:39.661 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-23T23:46:39.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 -- 172.21.15.99:0/3977870435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c107cc0 msgr2=0x7f2d8c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 --2- 172.21.15.99:0/3977870435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c1080e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009900 tx=0x7f2d7c02bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 -- 172.21.15.99:0/3977870435 shutdown_connections 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 --2- 172.21.15.99:0/3977870435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c109870 0x7f2d8c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 --2- 172.21.15.99:0/3977870435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 --2- 172.21.15.99:0/3977870435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c1080e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 -- 172.21.15.99:0/3977870435 >> 172.21.15.99:0/3977870435 conn(0x7f2d8c076c60 msgr2=0x7f2d8c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 -- 172.21.15.99:0/3977870435 shutdown_connections 2024-03-23T23:46:39.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.866+0000 7f2d93200640 1 -- 172.21.15.99:0/3977870435 wait complete. 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.867+0000 7f2d93200640 1 Processor -- start 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.867+0000 7f2d93200640 1 -- start start 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 0x7f2d8c1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c113980 con 0x7f2d8c107cc0 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c113800 con 0x7f2d8c109870 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d93200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c113b00 con 0x7f2d8c108eb0 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57304/0 (socket says 172.21.15.99:57304) 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d91200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 0x7f2d8c1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 -- 172.21.15.99:0/2908105491 learned_addr learned my addr 172.21.15.99:0/2908105491 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d83400640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 -- 172.21.15.99:0/2908105491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 msgr2=0x7f2d8c19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 -- 172.21.15.99:0/2908105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 msgr2=0x7f2d8c1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.868+0000 7f2d8be00640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 0x7f2d8c1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d8be00640 1 -- 172.21.15.99:0/2908105491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c1a5c70 con 0x7f2d8c107cc0 2024-03-23T23:46:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d91200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 0x7f2d8c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d83400640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d8be00640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c000c00 tx=0x7f2d7c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d89600640 1 -- 172.21.15.99:0/2908105491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c043020 con 0x7f2d8c107cc0 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d89600640 1 -- 172.21.15.99:0/2908105491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c0054e0 con 0x7f2d8c107cc0 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c1a5f00 con 0x7f2d8c107cc0 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d89600640 1 -- 172.21.15.99:0/2908105491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c003ce0 con 0x7f2d8c107cc0 2024-03-23T23:46:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.869+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c1a6360 con 0x7f2d8c107cc0 2024-03-23T23:46:39.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 Processor -- start 2024-03-23T23:46:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 -- start start 2024-03-23T23:46:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d89600640 1 -- 172.21.15.99:0/2908105491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2d7c046020 con 0x7f2d8c107cc0 2024-03-23T23:46:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc911c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc912e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc8762c0 con 0x5571fc911810 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc81ec90 con 0x5571fc9133a0 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.874+0000 7f2d96036740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc872360 con 0x5571fc9129e0 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53266/0 (socket says 172.21.15.99:53266) 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 -- 172.21.15.99:0/1205824316 learned_addr learned my addr 172.21.15.99:0/1205824316 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d8be00640 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc912e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d91200640 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc911c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 -- 172.21.15.99:0/1205824316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 msgr2=0x5571fc912e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc912e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 -- 172.21.15.99:0/1205824316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 msgr2=0x5571fc911c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc911c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.875+0000 7f2d83400640 1 -- 172.21.15.99:0/1205824316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571fc843080 con 0x5571fc9133a0 2024-03-23T23:46:39.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d83400640 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d740114f0 tx=0x7f2d740119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b51efeff378e1b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d81600640 1 -- 172.21.15.99:0/1205824316 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004020 con 0x5571fc9133a0 2024-03-23T23:46:39.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d81600640 1 -- 172.21.15.99:0/1205824316 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d740041c0 con 0x5571fc9133a0 2024-03-23T23:46:39.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d81600640 1 -- 172.21.15.99:0/1205824316 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400f8f0 con 0x5571fc9133a0 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d89600640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 0x7f2d700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.876+0000 7f2d89600640 1 -- 172.21.15.99:0/2908105491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d7c0f0b50 con 0x7f2d8c107cc0 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d91200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 0x7f2d700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 -- 172.21.15.99:0/1205824316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 msgr2=0x5571fc919c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d740114f0 tx=0x7f2d740119b0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d91200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 0x7f2d700a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5571fc9128e0 tx=0x7f2d840096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 -- 172.21.15.99:0/1205824316 shutdown_connections 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc919c30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc912e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 --2- 172.21.15.99:0/1205824316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc911c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 -- 172.21.15.99:0/1205824316 >> 172.21.15.99:0/1205824316 conn(0x5571fc90cc00 msgr2=0x5571fc90d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.877+0000 7f2d96036740 1 -- 172.21.15.99:0/1205824316 shutdown_connections 2024-03-23T23:46:39.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.878+0000 7f2d96036740 1 -- 172.21.15.99:0/1205824316 wait complete. 2024-03-23T23:46:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.878+0000 7f2d96036740 1 -- start start 2024-03-23T23:46:39.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.878+0000 7f2d96036740 1 Processor -- start 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9133a0 0x5571fc922f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc91a3b0 con 0x5571fc911810 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc8320d0 con 0x5571fc9129e0 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571fc91a530 con 0x5571fc9133a0 2024-03-23T23:46:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d96036740 10 client.0 _ll_register_callbacks cb 0x5571fc836e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d83400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d83400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53268/0 (socket says 172.21.15.99:53268) 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d83400640 1 -- 172.21.15.99:0/3200023405 learned_addr learned my addr 172.21.15.99:0/3200023405 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d8be00640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.879+0000 7f2d91200640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9133a0 0x5571fc922f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 -- 172.21.15.99:0/3200023405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9133a0 msgr2=0x5571fc922f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9133a0 0x5571fc922f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 -- 172.21.15.99:0/3200023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 msgr2=0x5571fc91bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 -- 172.21.15.99:0/3200023405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571fc58a3f0 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d83400640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d7401b040 tx=0x7f2d74009cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b86a232d9ed66d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74028020 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d74016040 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.880+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d740041c0 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d74033050 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 10 client.26243 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5571fc6547d0 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d740041c0 con 0x5571fc9129e0 2024-03-23T23:46:39.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2d74004b30 con 0x5571fc9129e0 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 10 client.26243 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 20 client.26243 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc927290 0x5571fc9296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 4 client.26243 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.881+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571fc883800 con 0x5571fc927290 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.882+0000 7f2d8be00640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc927290 0x5571fc9296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.882+0000 7f2d7a000640 10 client.26243 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.882+0000 7f2d8be00640 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc927290 0x5571fc9296d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57221a27233ee43f server_cookie=b8b7e94a018d180e in_seq=0 out_seq=0 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d7a000640 1 -- 172.21.15.99:0/3200023405 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5571fc883800 con 0x5571fc927290 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d7a000640 10 client.26243 handle_command_reply: tid=0 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 client.26243 shutdown 2024-03-23T23:46:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 msgr2=0x5571fc91bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d7401b040 tx=0x7f2d74009cc0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 shutdown_connections 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc927290 0x5571fc9296d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d7a000640 0 client.26243 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9133a0 0x5571fc922f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9129e0 0x5571fc91bff0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 --2- 172.21.15.99:0/3200023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 >> 172.21.15.99:0/3200023405 conn(0x5571fc90cc00 msgr2=0x5571fc90beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 shutdown_connections 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 1 -- 172.21.15.99:0/3200023405 wait complete. 2024-03-23T23:46:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.883+0000 7f2d96036740 20 client.26243 trim_cache size 0 max 16384 2024-03-23T23:46:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.898+0000 7f2d96036740 1 Processor -- start 2024-03-23T23:46:39.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 -- start start 2024-03-23T23:46:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 0x5571fc922f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc915fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1af030 con 0x5571fc9129e0 2024-03-23T23:46:39.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1b09b0 con 0x5571fc9133a0 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d96036740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1af690 con 0x5571fc911810 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d8be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57272/0 (socket says 172.21.15.99:57272) 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d8be00640 1 -- 172.21.15.99:0/1454780687 learned_addr learned my addr 172.21.15.99:0/1454780687 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.899+0000 7f2d91200640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc915fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d83400640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 0x5571fc922f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 -- 172.21.15.99:0/1454780687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 msgr2=0x5571fc915fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc915fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 -- 172.21.15.99:0/1454780687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 msgr2=0x5571fc922f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 0x5571fc922f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 -- 172.21.15.99:0/1454780687 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571fc9084e0 con 0x5571fc911810 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d91200640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc915fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:39.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d83400640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 0x5571fc922f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T23:46:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d8be00640 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006010 tx=0x7f2d7c0dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce95390b9356c437 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d80c00640 1 -- 172.21.15.99:0/1454780687 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0b9770 con 0x5571fc911810 2024-03-23T23:46:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d80c00640 1 -- 172.21.15.99:0/1454780687 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c004c80 con 0x5571fc911810 2024-03-23T23:46:39.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.900+0000 7f2d80c00640 1 -- 172.21.15.99:0/1454780687 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0bce30 con 0x5571fc911810 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 -- 172.21.15.99:0/1454780687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 msgr2=0x5571fc915a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006010 tx=0x7f2d7c0dd040 comp rx=0 tx=0).stop 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 -- 172.21.15.99:0/1454780687 shutdown_connections 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc915fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc9129e0 0x5571fc922f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 --2- 172.21.15.99:0/1454780687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc911810 0x5571fc915a80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 -- 172.21.15.99:0/1454780687 >> 172.21.15.99:0/1454780687 conn(0x5571fc90cc00 msgr2=0x5571fc9106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 -- 172.21.15.99:0/1454780687 shutdown_connections 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.901+0000 7f2d96036740 1 -- 172.21.15.99:0/1454780687 wait complete. 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.902+0000 7f2d96036740 1 -- start start 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.902+0000 7f2d96036740 1 Processor -- start 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc922150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1b0350 con 0x5571fc911810 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1afcf0 con 0x5571fc9133a0 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1ad1f0 con 0x5571fc9129e0 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d96036740 10 client.0 _ll_register_callbacks cb 0x7f2d84009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57278/0 (socket says 172.21.15.99:57278) 2024-03-23T23:46:39.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 -- 172.21.15.99:0/981251473 learned_addr learned my addr 172.21.15.99:0/981251473 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d8be00640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d91200640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc922150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 -- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 msgr2=0x5571fc922150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc922150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 -- 172.21.15.99:0/981251473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 msgr2=0x5571fc91ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.903+0000 7f2d83400640 1 -- 172.21.15.99:0/981251473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571fc922740 con 0x5571fc9129e0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d83400640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400f620 tx=0x7f2d74005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca5eb318f51a607 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74028070 con 0x5571fc9129e0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7c004c80 con 0x5571fc9129e0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d74005250 con 0x5571fc9129e0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7400f8a0 con 0x5571fc9129e0 2024-03-23T23:46:39.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d96036740 10 client.26284 fetch_fsmap learned FSMap version 27 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5571fc983050 con 0x5571fc9129e0 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7c004c80 con 0x5571fc9129e0 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.904+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f2d74004640 con 0x5571fc9129e0 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 10 client.26284 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 20 client.26284 populate_metadata read hostname 'smithi099' 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc9839d0 0x5571fc985df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 4 client.26284 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 --> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5571fc87db50 con 0x5571fc9839d0 2024-03-23T23:46:39.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d8be00640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc9839d0 0x5571fc985df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d8be00640 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc9839d0 0x5571fc985df0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48cb567518062e16 server_cookie=76fa2a75e80c6a65 in_seq=0 out_seq=0 2024-03-23T23:46:39.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.905+0000 7f2d7a000640 10 client.26284 ms_handle_connect on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d7a000640 1 -- 172.21.15.99:0/981251473 <== mds.0 v2:172.21.15.149:6838/2464794793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5571fc87db50 con 0x5571fc9839d0 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d7a000640 10 client.26284 handle_command_reply: tid=0 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 client.26284 shutdown 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 msgr2=0x5571fc91f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400f620 tx=0x7f2d74005f90 comp rx=0 tx=0).stop 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 shutdown_connections 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:6838/2464794793,v1:172.21.15.149:6839/2464794793] conn(0x5571fc9839d0 0x5571fc985df0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571fc9133a0 0x5571fc922150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5571fc9129e0 0x5571fc91f180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d7a000640 0 client.26284 ms_handle_reset on v2:172.21.15.149:6838/2464794793 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 --2- 172.21.15.99:0/981251473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5571fc911810 0x5571fc91ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 >> 172.21.15.99:0/981251473 conn(0x5571fc90cc00 msgr2=0x5571fc90b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 shutdown_connections 2024-03-23T23:46:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 1 -- 172.21.15.99:0/981251473 wait complete. 2024-03-23T23:46:39.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.906+0000 7f2d96036740 20 client.26284 trim_cache size 0 max 16384 2024-03-23T23:46:39.911 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 msgr2=0x7f2d700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 0x7f2d700a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5571fc9128e0 tx=0x7f2d840096a0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 msgr2=0x7f2d8c19e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c000c00 tx=0x7f2d7c02c6c0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 shutdown_connections 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f2d700a2800 0x7f2d700a4cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d8c109870 0x7f2d8c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d8c108eb0 0x7f2d8c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 --2- 172.21.15.99:0/2908105491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d8c107cc0 0x7f2d8c19e050 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 >> 172.21.15.99:0/2908105491 conn(0x7f2d8c076c60 msgr2=0x7f2d8c10c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 shutdown_connections 2024-03-23T23:46:39.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:39.907+0000 7f2d93200640 1 -- 172.21.15.99:0/2908105491 wait complete. 2024-03-23T23:46:40.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:39 smithi099 ceph-mon[31715]: pgmap v483: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-23T23:46:40.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:39 smithi149 ceph-mon[36083]: pgmap v483: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-23T23:46:40.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:39 smithi174 ceph-mon[37002]: pgmap v483: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-23T23:46:40.933 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-23T23:46:40.934 INFO:tasks.fwd_scrub:done joining 2024-03-23T23:46:40.934 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-23T23:46:40.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:46:41.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 -- 172.21.15.99:0/3534802714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 msgr2=0x7f38a0109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 --2- 172.21.15.99:0/3534802714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0109640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3898009900 tx=0x7f389802bc50 comp rx=0 tx=0).stop 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 -- 172.21.15.99:0/3534802714 shutdown_connections 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 --2- 172.21.15.99:0/3534802714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0109640 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 --2- 172.21.15.99:0/3534802714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 --2- 172.21.15.99:0/3534802714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 0x7f38a0104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 -- 172.21.15.99:0/3534802714 >> 172.21.15.99:0/3534802714 conn(0x7f38a00fb980 msgr2=0x7f38a00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 -- 172.21.15.99:0/3534802714 shutdown_connections 2024-03-23T23:46:41.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.151+0000 7f389fe00640 1 -- 172.21.15.99:0/3534802714 wait complete. 2024-03-23T23:46:41.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 Processor -- start 2024-03-23T23:46:41.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 -- start start 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 0x7f38a0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a00fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a010d1f0 con 0x7f38a0101f90 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a010d070 con 0x7f38a01071f0 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a010d370 con 0x7f38a01048c0 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.153+0000 7f389f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53304/0 (socket says 172.21.15.99:53304) 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 -- 172.21.15.99:0/733604583 learned_addr learned my addr 172.21.15.99:0/733604583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389ea00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 0x7f38a0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f3896000640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a00fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 -- 172.21.15.99:0/733604583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 msgr2=0x7f38a00fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a00fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 -- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 msgr2=0x7f38a0101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 0x7f38a0101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 -- 172.21.15.99:0/733604583 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0100d10 con 0x7f38a01071f0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f3896000640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.154+0000 7f389f400640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3898036040 tx=0x7f389802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.155+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898041070 con 0x7f38a01071f0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.155+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3898004450 con 0x7f38a01071f0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.155+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898034440 con 0x7f38a01071f0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.155+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a01a1900 con 0x7f38a01071f0 2024-03-23T23:46:41.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.155+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a01a1e00 con 0x7f38a01071f0 2024-03-23T23:46:41.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.156+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a00fd470 con 0x7f38a01071f0 2024-03-23T23:46:41.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.158+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3898044070 con 0x7f38a01071f0 2024-03-23T23:46:41.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.160+0000 7f3897e00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 0x7f38840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.160+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38980f05a0 con 0x7f38a01071f0 2024-03-23T23:46:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.161+0000 7f389ea00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 0x7f38840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.161+0000 7f389ea00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 0x7f38840a4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3890005fd0 tx=0x7f3890009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:41.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.163+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38980bcf70 con 0x7f38a01071f0 2024-03-23T23:46:41.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.365+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38a00622d0 con 0x7f38a01071f0 2024-03-23T23:46:41.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.367+0000 7f3897e00640 1 -- 172.21.15.99:0/733604583 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7f38980c1010 con 0x7f38a01071f0 2024-03-23T23:46:41.367 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:46:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 msgr2=0x7f38840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 0x7f38840a4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3890005fd0 tx=0x7f3890009500 comp rx=0 tx=0).stop 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 msgr2=0x7f38a0100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3898036040 tx=0x7f389802c650 comp rx=0 tx=0).stop 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 shutdown_connections 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f38840a2880 0x7f38840a4d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38a01071f0 0x7f38a0100480 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a01048c0 0x7f38a00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 --2- 172.21.15.99:0/733604583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a0101f90 0x7f38a0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.369+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 >> 172.21.15.99:0/733604583 conn(0x7f38a00fb980 msgr2=0x7f38a01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:41.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.370+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 shutdown_connections 2024-03-23T23:46:41.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.370+0000 7f389fe00640 1 -- 172.21.15.99:0/733604583 wait complete. 2024-03-23T23:46:41.387 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:46:41.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 -- 172.21.15.99:0/3852232543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2501038e0 msgr2=0x7ff250103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 --2- 172.21.15.99:0/3852232543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2501038e0 0x7ff250103d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff240009900 tx=0x7ff24002bc30 comp rx=0 tx=0).stop 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 -- 172.21.15.99:0/3852232543 shutdown_connections 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 --2- 172.21.15.99:0/3852232543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff250105490 0x7ff25010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 --2- 172.21.15.99:0/3852232543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250104ad0 0x7ff250104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 --2- 172.21.15.99:0/3852232543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2501038e0 0x7ff250103d00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 -- 172.21.15.99:0/3852232543 >> 172.21.15.99:0/3852232543 conn(0x7ff2500fee50 msgr2=0x7ff250101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.587+0000 7ff255c00640 1 -- 172.21.15.99:0/3852232543 shutdown_connections 2024-03-23T23:46:41.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.588+0000 7ff255c00640 1 -- 172.21.15.99:0/3852232543 wait complete. 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.588+0000 7ff255c00640 1 Processor -- start 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 -- start start 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 0x7ff25019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 0x7ff2501a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010f580 con 0x7ff250105490 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010f400 con 0x7ff250104ad0 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff255c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25010f700 con 0x7ff2501038e0 2024-03-23T23:46:41.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff247e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 0x7ff25019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff24ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff24ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57304/0 (socket says 172.21.15.99:57304) 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff24ea00640 1 -- 172.21.15.99:0/1576300325 learned_addr learned my addr 172.21.15.99:0/1576300325 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.589+0000 7ff24f400640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 0x7ff2501a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 -- 172.21.15.99:0/1576300325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 msgr2=0x7ff25019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 0x7ff25019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 -- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 msgr2=0x7ff2501a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 0x7ff2501a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 -- 172.21.15.99:0/1576300325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2501a1860 con 0x7ff2501038e0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24f400640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 0x7ff2501a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff247e00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 0x7ff25019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff24ea00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff240000c00 tx=0x7ff24002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240043020 con 0x7ff2501038e0 2024-03-23T23:46:41.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.590+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2400054e0 con 0x7ff2501038e0 2024-03-23T23:46:41.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.591+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2501a1af0 con 0x7ff2501038e0 2024-03-23T23:46:41.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.591+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240003c40 con 0x7ff2501038e0 2024-03-23T23:46:41.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.591+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2501a2040 con 0x7ff2501038e0 2024-03-23T23:46:41.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.592+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2500688e0 con 0x7ff2501038e0 2024-03-23T23:46:41.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.595+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff240046020 con 0x7ff2501038e0 2024-03-23T23:46:41.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.596+0000 7ff247400640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 0x7ff2340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:41.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.596+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2400f0bc0 con 0x7ff2501038e0 2024-03-23T23:46:41.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.597+0000 7ff247e00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 0x7ff2340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:41.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.597+0000 7ff247e00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 0x7ff2340a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff238009fc0 tx=0x7ff238005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:41.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.600+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff24004d020 con 0x7ff2501038e0 2024-03-23T23:46:41.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.812+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2500622a0 con 0x7ff2501038e0 2024-03-23T23:46:41.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.813+0000 7ff247400640 1 -- 172.21.15.99:0/1576300325 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2400bdbe0 con 0x7ff2501038e0 2024-03-23T23:46:41.813 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 msgr2=0x7ff2340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 0x7ff2340a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff238009fc0 tx=0x7ff238005f90 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 msgr2=0x7ff250199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff240000c00 tx=0x7ff24002c650 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 shutdown_connections 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7ff2340a2800 0x7ff2340a4cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250105490 0x7ff2501a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff250104ad0 0x7ff25019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 --2- 172.21.15.99:0/1576300325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2501038e0 0x7ff250199c50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.815+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 >> 172.21.15.99:0/1576300325 conn(0x7ff2500fee50 msgr2=0x7ff250107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:41.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.816+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 shutdown_connections 2024-03-23T23:46:41.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:41.816+0000 7ff255c00640 1 -- 172.21.15.99:0/1576300325 wait complete. 2024-03-23T23:46:41.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 -- 172.21.15.99:0/3993750283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260810db70 msgr2=0x7f260810ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 --2- 172.21.15.99:0/3993750283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260810db70 0x7f260810ff60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2600009ca0 tx=0x7f260002c0f0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 -- 172.21.15.99:0/3993750283 shutdown_connections 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 --2- 172.21.15.99:0/3993750283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f260810db70 0x7f260810ff60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 --2- 172.21.15.99:0/3993750283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f2608074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 --2- 172.21.15.99:0/3993750283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26080756e0 0x7f2608073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 -- 172.21.15.99:0/3993750283 >> 172.21.15.99:0/3993750283 conn(0x7f26080fcb10 msgr2=0x7f26080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:42.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 -- 172.21.15.99:0/3993750283 shutdown_connections 2024-03-23T23:46:42.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.025+0000 7f2607e00640 1 -- 172.21.15.99:0/3993750283 wait complete. 2024-03-23T23:46:42.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.026+0000 7f2607e00640 1 Processor -- start 2024-03-23T23:46:42.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.026+0000 7f2607e00640 1 -- start start 2024-03-23T23:46:42.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26080756e0 0x7f260819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260810db70 0x7f26081a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26081137f0 con 0x7f260810db70 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608113670 con 0x7f2608074100 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608113970 con 0x7f26080756e0 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53336/0 (socket says 172.21.15.99:53336) 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 -- 172.21.15.99:0/184477800 learned_addr learned my addr 172.21.15.99:0/184477800 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606000640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26080756e0 0x7f260819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2607400640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260810db70 0x7f26081a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 -- 172.21.15.99:0/184477800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26080756e0 msgr2=0x7f260819e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26080756e0 0x7f260819e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 -- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260810db70 msgr2=0x7f26081a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260810db70 0x7f26081a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.027+0000 7f2606a00640 1 -- 172.21.15.99:0/184477800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26081a5a40 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f2606a00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9f0 tx=0x7f25f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800cd80 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26081a5d30 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25f8010910 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8018990 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.028+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081a61d0 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.029+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2608103d40 con 0x7f2608074100 2024-03-23T23:46:42.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.031+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f25f8010ac0 con 0x7f2608074100 2024-03-23T23:46:42.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.032+0000 7f25ff400640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 0x7f25ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.032+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f25f80ceee0 con 0x7f2608074100 2024-03-23T23:46:42.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.033+0000 7f2606000640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 0x7f25ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.033+0000 7f2606000640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 0x7f25ec0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25f40095f0 tx=0x7f25f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.036+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25f809bf00 con 0x7f2608074100 2024-03-23T23:46:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:41 smithi099 ceph-mon[31715]: pgmap v484: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:41 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/733604583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:46:42.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:41 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1576300325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:46:42.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.235+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2608062340 con 0x7f2608074100 2024-03-23T23:46:42.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.236+0000 7f25ff400640 1 -- 172.21.15.99:0/184477800 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25f809b800 con 0x7f2608074100 2024-03-23T23:46:42.236 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:46:42.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 msgr2=0x7f25ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 0x7f25ec0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25f40095f0 tx=0x7f25f4005f90 comp rx=0 tx=0).stop 2024-03-23T23:46:42.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 msgr2=0x7f260819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9f0 tx=0x7f25f800eeb0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 shutdown_connections 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f25ec0a2800 0x7f25ec0a4cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260810db70 0x7f26081a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26080756e0 0x7f260819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 --2- 172.21.15.99:0/184477800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2608074100 0x7f260819de20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.238+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 >> 172.21.15.99:0/184477800 conn(0x7f26080fcb10 msgr2=0x7f2608100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.239+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 shutdown_connections 2024-03-23T23:46:42.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.239+0000 7f2607e00640 1 -- 172.21.15.99:0/184477800 wait complete. 2024-03-23T23:46:42.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:46:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:41 smithi149 ceph-mon[36083]: pgmap v484: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:41 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/733604583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:46:42.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:41 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1576300325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:46:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:41 smithi174 ceph-mon[37002]: pgmap v484: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:41 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/733604583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:46:42.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:41 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1576300325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:46:42.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 -- 172.21.15.99:0/2968486622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 msgr2=0x7f538c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 --2- 172.21.15.99:0/2968486622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c104e10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5374009900 tx=0x7f537402bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 -- 172.21.15.99:0/2968486622 shutdown_connections 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 --2- 172.21.15.99:0/2968486622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 0x7f538c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 --2- 172.21.15.99:0/2968486622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c104e10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 --2- 172.21.15.99:0/2968486622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 -- 172.21.15.99:0/2968486622 >> 172.21.15.99:0/2968486622 conn(0x7f538c0fed10 msgr2=0x7f538c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 -- 172.21.15.99:0/2968486622 shutdown_connections 2024-03-23T23:46:42.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.466+0000 7f538be00640 1 -- 172.21.15.99:0/2968486622 wait complete. 2024-03-23T23:46:42.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 Processor -- start 2024-03-23T23:46:42.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 -- start start 2024-03-23T23:46:42.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 0x7f538c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10f470 con 0x7f538c105350 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10f2f0 con 0x7f538c1037a0 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10f5f0 con 0x7f538c104990 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:53346/0 (socket says 172.21.15.99:53346) 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f538aa00640 1 -- 172.21.15.99:0/1915346229 learned_addr learned my addr 172.21.15.99:0/1915346229 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.468+0000 7f5383e00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538b400640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 0x7f538c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 -- 172.21.15.99:0/1915346229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 msgr2=0x7f538c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 -- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 msgr2=0x7f538c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 0x7f538c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 -- 172.21.15.99:0/1915346229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c1a16a0 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538aa00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f537c00e9c0 tx=0x7f537c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00cd80 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f537c0035f0 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c1a1990 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.469+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c010690 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.470+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1a1e60 con 0x7f538c1037a0 2024-03-23T23:46:42.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.471+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c10aa60 con 0x7f538c1037a0 2024-03-23T23:46:42.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.473+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f537c0037a0 con 0x7f538c1037a0 2024-03-23T23:46:42.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.474+0000 7f5383400640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 0x7f53700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.474+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f537c0cee40 con 0x7f538c1037a0 2024-03-23T23:46:42.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.474+0000 7f5383e00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 0x7f53700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.475+0000 7f5383e00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 0x7f53700a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f537402c250 tx=0x7f537402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.478+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f537c09be60 con 0x7f538c1037a0 2024-03-23T23:46:42.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.675+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f538c0622d0 con 0x7f538c1037a0 2024-03-23T23:46:42.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.675+0000 7f5383400640 1 -- 172.21.15.99:0/1915346229 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f537c09b760 con 0x7f538c1037a0 2024-03-23T23:46:42.676 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 msgr2=0x7f53700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 0x7f53700a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f537402c250 tx=0x7f537402cf90 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 msgr2=0x7f538c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f537c00e9c0 tx=0x7f537c00ee80 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 shutdown_connections 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f53700a2800 0x7f53700a4cc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f538c105350 0x7f538c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f538c104990 0x7f538c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 --2- 172.21.15.99:0/1915346229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f538c1037a0 0x7f538c199a80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 >> 172.21.15.99:0/1915346229 conn(0x7f538c0fed10 msgr2=0x7f538c107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:42.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 shutdown_connections 2024-03-23T23:46:42.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.678+0000 7f538be00640 1 -- 172.21.15.99:0/1915346229 wait complete. 2024-03-23T23:46:42.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 -- 172.21.15.99:0/239244520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa94100640 msgr2=0x7faa94100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 --2- 172.21.15.99:0/239244520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa94100640 0x7faa94100a60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa84009900 tx=0x7faa8402bc40 comp rx=0 tx=0).stop 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 -- 172.21.15.99:0/239244520 shutdown_connections 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 --2- 172.21.15.99:0/239244520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa941021f0 0x7faa94108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 --2- 172.21.15.99:0/239244520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94101830 0x7faa94101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 --2- 172.21.15.99:0/239244520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa94100640 0x7faa94100a60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 -- 172.21.15.99:0/239244520 >> 172.21.15.99:0/239244520 conn(0x7faa940fbbd0 msgr2=0x7faa940fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 -- 172.21.15.99:0/239244520 shutdown_connections 2024-03-23T23:46:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.908+0000 7faa9bc00640 1 -- 172.21.15.99:0/239244520 wait complete. 2024-03-23T23:46:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 Processor -- start 2024-03-23T23:46:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 -- start start 2024-03-23T23:46:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 0x7faa94197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 0x7faa9419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9410c2a0 con 0x7faa94101830 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9410c120 con 0x7faa941021f0 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.910+0000 7faa9bc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9410c420 con 0x7faa94100640 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 0x7faa9419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:57372/0 (socket says 172.21.15.99:57372) 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99200640 1 -- 172.21.15.99:0/84871918 learned_addr learned my addr 172.21.15.99:0/84871918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa8be00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 0x7faa94197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 -- 172.21.15.99:0/84871918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 msgr2=0x7faa94197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 -- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 msgr2=0x7faa94197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 0x7faa94197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 -- 172.21.15.99:0/84871918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9419f660 con 0x7faa941021f0 2024-03-23T23:46:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99200640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:46:42.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa8be00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 0x7faa94197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:46:42.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa99c00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 0x7faa9419eff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e940 tx=0x7faa8c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.911+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004d20 con 0x7faa941021f0 2024-03-23T23:46:42.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.912+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9419f950 con 0x7faa941021f0 2024-03-23T23:46:42.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.912+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa8c005020 con 0x7faa941021f0 2024-03-23T23:46:42.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.912+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c0116b0 con 0x7faa941021f0 2024-03-23T23:46:42.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.912+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9419fe50 con 0x7faa941021f0 2024-03-23T23:46:42.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.913+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa941a00e0 con 0x7faa941021f0 2024-03-23T23:46:42.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.915+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faa8c0054d0 con 0x7faa941021f0 2024-03-23T23:46:42.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.916+0000 7faa92000640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 0x7faa780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:46:42.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.917+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faa8c0cef60 con 0x7faa941021f0 2024-03-23T23:46:42.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.917+0000 7faa99200640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 0x7faa780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:46:42.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.917+0000 7faa99200640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 0x7faa780a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa84005ab0 tx=0x7faa8402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:46:42.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:42.922+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa8c016030 con 0x7faa941021f0 2024-03-23T23:46:43.113 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:42 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/184477800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:46:43.114 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:42 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1915346229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:46:43.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.180+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7faa941a02b0 con 0x7faa941021f0 2024-03-23T23:46:43.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.190+0000 7faa92000640 1 -- 172.21.15.99:0/84871918 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+739827 (secure 0 0 0) 0x7faa8c09bf80 con 0x7faa941021f0 2024-03-23T23:46:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 18, 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "active_gid": 14152, 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "active_name": "x", 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addrs": { 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3957482815 2024-03-23T23:46:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3957482815 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addr": "172.21.15.99:6801/3957482815", 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "active_change": "2024-03-23T23:33:06.585700+0000", 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "active_mgr_features": 4540701547738038271, 2024-03-23T23:46:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "available": true, 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "standbys": [ 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 24109, 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "z", 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540701547738038271, 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-23T23:46:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-23T23:46:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T23:46:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-23T23:46:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-23T23:46:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-23T23:46:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T23:46:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-23T23:46:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-03-23T23:46:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-23T23:46:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T23:46:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-23T23:46:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T23:46:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T23:46:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-23T23:46:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T23:46:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-23T23:46:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-23T23:46:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-23T23:46:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T23:46:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T23:46:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-23T23:46:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-23T23:46:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-23T23:46:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-23T23:46:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T23:46:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T23:46:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-23T23:46:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-23T23:46:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.313 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.316 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.321 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.322 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.324 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.325 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.326 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.327 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-23T23:46:43.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.330 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T23:46:43.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T23:46:43.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.335 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.336 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.336 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.341 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.342 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T23:46:43.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T23:46:43.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.353 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.359 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T23:46:43.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.362 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.364 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.375 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.379 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.380 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.381 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.384 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.386 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.388 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.391 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:43.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.398 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.403 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.405 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:43.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.411 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-23T23:46:43.412 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.413 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:43.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.420 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.422 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-23T23:46:43.425 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-23T23:46:43.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.429 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.434 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:43.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.439 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-23T23:46:43.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-23T23:46:43.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.446 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.454 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-23T23:46:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-23T23:46:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-23T23:46:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-23T23:46:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-23T23:46:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-23T23:46:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-23T23:46:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T23:46:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.554 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.555 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.556 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.560 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.561 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.562 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.568 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.569 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-23T23:46:43.572 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.576 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.580 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.582 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.583 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-23T23:46:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-23T23:46:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-23T23:46:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-23T23:46:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-23T23:46:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 24116, 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "y", 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540701547738038271, 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-23T23:46:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-23T23:46:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-23T23:46:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-23T23:46:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-23T23:46:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-03-23T23:46:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.696 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.697 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.699 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-23T23:46:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-23T23:46:43.701 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-23T23:46:43.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-23T23:46:43.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.706 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-23T23:46:43.707 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-23T23:46:43.708 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-23T23:46:43.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.712 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.714 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.717 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.719 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.722 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.723 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.725 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.725 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T23:46:43.725 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.725 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.727 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.728 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.729 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.730 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-23T23:46:43.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T23:46:43.732 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.734 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.735 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.737 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.738 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.739 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.740 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.743 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.744 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.747 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.749 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.750 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T23:46:43.753 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.756 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-23T23:46:43.759 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-23T23:46:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.762 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.765 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.767 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.768 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.769 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.770 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.772 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.773 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.774 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.775 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.777 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.779 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.780 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.781 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.783 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.784 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.785 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T23:46:43.786 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T23:46:43.787 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.795 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.797 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.798 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.799 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.802 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.803 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.804 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.805 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.806 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.807 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-23T23:46:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-23T23:46:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T23:46:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-23T23:46:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-23T23:46:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.872 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-23T23:46:43.874 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.875 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.876 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.877 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.879 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-23T23:46:43.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-23T23:46:43.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.882 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.883 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.884 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.885 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.886 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.887 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.888 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.891 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.892 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.894 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.896 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.897 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.898 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.899 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.900 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.901 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.904 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.907 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.908 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.910 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.912 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.914 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.915 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.916 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.918 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.919 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.922 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.925 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.926 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.928 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.928 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.929 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.930 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.931 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.932 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.937 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.938 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.940 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.941 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.943 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-23T23:46:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-23T23:46:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.991 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.992 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-23T23:46:43.993 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.994 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.995 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.996 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:43.997 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-23T23:46:43.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-23T23:46:43.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.002 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.003 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.004 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.009 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-23T23:46:44.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.011 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.013 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.014 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.015 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.017 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.018 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.019 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.020 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.023 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.024 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.025 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.026 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.027 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-23T23:46:44.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.029 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.031 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.032 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.033 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.035 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.036 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-23T23:46:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-23T23:46:44.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-23T23:46:44.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.043 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-23T23:46:44.044 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.045 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.046 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.047 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.048 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.051 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.052 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.054 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.056 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.058 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.059 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.060 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.061 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.062 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.063 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.064 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-23T23:46:44.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.068 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-23T23:46:44.070 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.071 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.074 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.076 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.078 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.079 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T23:46:44.080 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "modules": [ 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-23T23:46:44.081 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "iostat", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nfs", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "restful" 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.082 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.084 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.085 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.087 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.088 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.090 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.091 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.093 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T23:46:44.094 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.095 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-23T23:46:44.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-23T23:46:44.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-23T23:46:44.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-23T23:46:44.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.100 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.102 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.103 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.104 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.105 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.107 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.108 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.109 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.110 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "read", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap-read" 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.111 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.114 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.115 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.118 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-23T23:46:44.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T23:46:44.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm_log_destination": { 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm_log_destination", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.121 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "file", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "file,syslog", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "syslog" 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.123 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-23T23:46:44.124 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-23T23:46:44.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.126 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.127 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.128 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.130 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.134 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.135 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.136 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-23T23:46:44.137 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-23T23:46:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.139 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.141 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-23T23:46:44.143 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-23T23:46:44.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.146 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.149 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.150 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.151 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.152 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.153 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.154 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.159 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.162 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.164 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-23T23:46:44.165 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-23T23:46:44.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.168 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.172 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.174 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.175 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.176 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.177 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.179 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-23T23:46:44.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.181 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.182 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.184 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.186 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.189 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T23:46:44.190 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T23:46:44.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.195 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.197 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.199 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.201 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.204 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.205 INFO:teuthology.orchestra.run.smithi099.stdout: "MULTICLUSTER_CONFIG": { 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "{}", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.207 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T23:46:44.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T23:46:44.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T23:46:44.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.215 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.223 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T23:46:44.224 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T23:46:44.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-23T23:46:44.230 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.232 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.235 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.240 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:44.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.243 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.247 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.249 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.250 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T23:46:44.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T23:46:44.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.268 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.269 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.274 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.276 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:44.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T23:46:44.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.285 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.287 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.288 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.290 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-23T23:46:44.292 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.293 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.296 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.298 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.300 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.301 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.306 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-23T23:46:44.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-23T23:46:44.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-23T23:46:44.310 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.311 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.314 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.321 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.322 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.324 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.326 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.328 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.330 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.331 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.334 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.337 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.337 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.338 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.339 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.340 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.341 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.342 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.343 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.344 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-23T23:46:44.345 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.346 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.347 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.349 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:44.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-23T23:46:44.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T23:46:44.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.355 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.360 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.364 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.380 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-23T23:46:44.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-23T23:46:44.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-23T23:46:44.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.390 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.394 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.397 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T23:46:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.415 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.417 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.418 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-23T23:46:44.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.421 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-23T23:46:44.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.426 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.428 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.429 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.431 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.436 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T23:46:44.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.446 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.447 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.456 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.461 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.468 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.469 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.471 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.473 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.474 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.478 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "periodic_async_work": { 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "periodic_async_work", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Periodically check for async work", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.481 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.486 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.491 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "services": { 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard": "https://172.21.15.99:8443/" 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "always_on_modules": { 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "octopus": [ 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-23T23:46:44.494 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pacific": [ 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-23T23:46:44.495 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "quincy": [ 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-23T23:46:44.496 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "reef": [ 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "squid": [ 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-23T23:46:44.497 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "last_failure_osd_epoch": 3, 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "active_clients": [ 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-23T23:46:44.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 559950094 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "libcephsqlite", 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 769837313 2024-03-23T23:46:44.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 475807309 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-23T23:46:44.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 348256676 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-23T23:46:44.501 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:42 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/184477800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:42 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1915346229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:43 smithi174 ceph-mon[37002]: pgmap v485: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:43 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/84871918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:42 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/184477800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:42 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1915346229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:43 smithi149 ceph-mon[36083]: pgmap v485: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:44.503 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:43 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/84871918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T23:46:44.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.198+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 msgr2=0x7faa780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:44.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.198+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 0x7faa780a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa84005ab0 tx=0x7faa8402cf90 comp rx=0 tx=0).stop 2024-03-23T23:46:44.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 msgr2=0x7faa9419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:46:44.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 0x7faa9419eff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e940 tx=0x7faa8c00ee00 comp rx=0 tx=0).stop 2024-03-23T23:46:44.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 shutdown_connections 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faa780a2800 0x7faa780a4cc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa941021f0 0x7faa9419eff0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa94101830 0x7faa94197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 --2- 172.21.15.99:0/84871918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa94100640 0x7faa94197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 >> 172.21.15.99:0/84871918 conn(0x7faa940fbbd0 msgr2=0x7faa940fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 shutdown_connections 2024-03-23T23:46:44.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:46:43.199+0000 7faa9bc00640 1 -- 172.21.15.99:0/84871918 wait complete. 2024-03-23T23:46:44.511 DEBUG:tasks.check_counter:Getting stats from a 2024-03-23T23:46:44.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-23T23:46:44.511 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:43 smithi099 ceph-mon[31715]: pgmap v485: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:44.511 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:43 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/84871918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T23:46:44.970 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:46:46.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:45 smithi099 ceph-mon[31715]: pgmap v486: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:46.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:45 smithi149 ceph-mon[36083]: pgmap v486: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:46.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:45 smithi174 ceph-mon[37002]: pgmap v486: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:47.373 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 33, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 37, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 8359, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 56336, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 15, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.025474431, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.004904081, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.013238273, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000719785, 2024-03-23T23:46:47.374 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 33, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.035901121, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001087912 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 18, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000005213, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000289 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 736 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 205, 2024-03-23T23:46:47.375 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 190, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 521901, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 106711, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.047968548, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.012767266, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.042496744, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.004024019, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 194, 2024-03-23T23:46:47.376 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.056780482, 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000292682 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 85, 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000020003, 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000235 2024-03-23T23:46:47.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 486944, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 63440 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 712, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 566, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 215342, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 453976, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 18, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.125985160, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.055244225, 2024-03-23T23:46:47.378 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.079947034, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.002665380, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 562, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.051375587, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000091415 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1172, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000398894, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000340 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 206672 2024-03-23T23:46:47.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 39, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002151973, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000055178 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.380 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000008183, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000004091 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 1, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 18, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 2, 2024-03-23T23:46:47.382 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 16, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 8, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 11, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 6, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 1, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 4, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 2000, 2024-03-23T23:46:47.383 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 2, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:46:47.384 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:46:47.385 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 17, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 10, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 10, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-03-23T23:46:47.386 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:46:47.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 5, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 5, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-03-23T23:46:47.388 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195730, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007386463, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001846615 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 22, 2024-03-23T23:46:47.389 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 28, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 6, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 21, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 26, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 5, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 18, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 26, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 8, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 50936, 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 223612 2024-03-23T23:46:47.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 1, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 1, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 7, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.391 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-03-23T23:46:47.393 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.394 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.396 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-03-23T23:46:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-03-23T23:46:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-03-23T23:46:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-03-23T23:46:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-03-23T23:46:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-03-23T23:46:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-03-23T23:46:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-03-23T23:46:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-03-23T23:46:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 184, 2024-03-23T23:46:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-03-23T23:46:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-23T23:46:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:46:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:46:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:46:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:46:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:46:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:46:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-23T23:46:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:46:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-23T23:46:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 238203, 2024-03-23T23:46:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 77, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-23T23:46:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7720, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 162, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-23T23:46:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 87885, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 346, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-03-23T23:46:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-03-23T23:46:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-03-23T23:46:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 40, 2024-03-23T23:46:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 11728, 2024-03-23T23:46:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 33, 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 40, 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 40, 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.347991842, 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008699796 2024-03-23T23:46:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 40, 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 62 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 40, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 40, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-03-23T23:46:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 4, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 7, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-03-23T23:46:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-03-23T23:46:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-03-23T23:46:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-03-23T23:46:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-03-23T23:46:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 87, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 8, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 8, 2024-03-23T23:46:47.431 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 14, 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 1 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-03-23T23:46:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 946, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 650187, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 946, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 946, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 650187, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-03-23T23:46:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 40, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 11728, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 40, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 11728, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 40, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 40, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 40, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 40, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56519b0b45a0": { 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 5, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1426, 2024-03-23T23:46:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 5, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1426, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:47.443 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:48.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:47 smithi099 ceph-mon[31715]: pgmap v487: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:48.355 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-23T23:46:48.355 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-23T23:46:48.355 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-23T23:46:48.355 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=17 2024-03-23T23:46:48.355 DEBUG:tasks.check_counter:Getting stats from d 2024-03-23T23:46:48.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-23T23:46:48.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:47 smithi149 ceph-mon[36083]: pgmap v487: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:48.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:47 smithi174 ceph-mon[37002]: pgmap v487: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:48.851 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:46:50.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:49 smithi099 ceph-mon[31715]: pgmap v488: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:46:50.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:49 smithi149 ceph-mon[36083]: pgmap v488: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:46:50.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:49 smithi174 ceph-mon[37002]: pgmap v488: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:46:51.244 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:51.244 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:46:51.244 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-03-23T23:46:51.244 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003777581, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T23:46:51.245 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000576196, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:46:51.246 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 80, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3184, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 91696, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 3, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.014434439, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.009638345, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003278387, 2024-03-23T23:46:51.247 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 79, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007479192, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000094673 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 91696 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:46:51.248 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 106, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 103, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 444336, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 61088, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.029998431, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.011660415, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.017012245, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.003533721, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 102, 2024-03-23T23:46:51.249 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007554532, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000074064 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 444336, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 61088 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.250 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:46:51.251 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-23T23:46:51.252 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-23T23:46:51.253 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148823, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 27, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-23T23:46:51.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:46:51.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:51.256 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 107, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 429413, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 107, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 107, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 429413, 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:51.257 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:51.258 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:51.258 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:51.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:51.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:51.258 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:51 smithi099 ceph-mon[31715]: pgmap v489: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:52.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:51 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:46:52.325 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-23T23:46:52.325 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-23T23:46:52.325 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-23T23:46:52.325 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-23T23:46:52.325 DEBUG:tasks.check_counter:Getting stats from g 2024-03-23T23:46:52.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-23T23:46:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:51 smithi149 ceph-mon[36083]: pgmap v489: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:52.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:51 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:46:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:51 smithi174 ceph-mon[37002]: pgmap v489: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:46:52.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:51 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T23:46:52.792 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:46:53.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:46:53.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:46:53.273 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:52 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:46:53.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:46:53.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:46:53.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:52 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:46:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T23:46:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T23:46:53.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:52 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' 2024-03-23T23:46:54.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:53 smithi099 ceph-mon[31715]: pgmap v490: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:54.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:53 smithi149 ceph-mon[36083]: pgmap v490: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:54.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:53 smithi174 ceph-mon[37002]: pgmap v490: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 881, 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 660, 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 718376, 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 222743, 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-03-23T23:46:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.149689923, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.059142937, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.097652216, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.006809219, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 660, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.083867367, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000127071 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1200, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000370050, 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000308 2024-03-23T23:46:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 491552, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 65312 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 40, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 115, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 16510, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 348401, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 17, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.031965830, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.014671456, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.014605546, 2024-03-23T23:46:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000814648, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 112, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.028350106, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000253125 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 50, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000012898, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000257 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 287408 2024-03-23T23:46:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 41, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 43, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 16239, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 132435, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 18, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.017571655, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.002857397, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.013019688, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000739335, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 41, 2024-03-23T23:46:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.044073269, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001074957 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 42, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000010879, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000259 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-03-23T23:46:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 36, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002573483, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000071485 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000011593, 2024-03-23T23:46:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000005796 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 1, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 1, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000165233, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000165233 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-03-23T23:46:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 18, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 2, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 16, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 8, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 1, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 1, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 10, 2024-03-23T23:46:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 6, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 4, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:46:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:46:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 17, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 10, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 10, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-03-23T23:46:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-03-23T23:46:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 5, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 5, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-03-23T23:46:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195730, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007514666, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001878666 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-03-23T23:46:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 22, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 28, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 6, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 21, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 26, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 5, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 18, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 26, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 8, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 1, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 1, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 50688, 2024-03-23T23:46:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 223612 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 1, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 1, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 7, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-03-23T23:46:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000165233, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000165233 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-03-23T23:46:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-03-23T23:46:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-03-23T23:46:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-03-23T23:46:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 2, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 1, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 1, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 180, 2024-03-23T23:46:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:46:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:46:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 270343, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 89, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-23T23:46:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7720, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 162, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 88237, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 346, 2024-03-23T23:46:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 40, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 11728, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 33, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 40, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 40, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.283993350, 2024-03-23T23:46:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007099833 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 40, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 62 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 40, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 40, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-03-23T23:46:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 4, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 7, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-03-23T23:46:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-03-23T23:46:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 87, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-03-23T23:46:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 2, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 9, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 9, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 14, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 1 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-03-23T23:46:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 958, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 654337, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 958, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 958, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 654337, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 40, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 11728, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 40, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 11728, 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 40, 2024-03-23T23:46:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 40, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 40, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 40, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-23T23:46:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x558a2bebe5a0": { 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 5, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1426, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 5, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1426, 2024-03-23T23:46:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:55.264 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:46:56.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:55 smithi099 ceph-mon[31715]: pgmap v491: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:56.332 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-23T23:46:56.332 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-23T23:46:56.332 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-23T23:46:56.332 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=17 2024-03-23T23:46:56.332 DEBUG:tasks.check_counter:Getting stats from j 2024-03-23T23:46:56.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-23T23:46:56.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:55 smithi149 ceph-mon[36083]: pgmap v491: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:56.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:55 smithi174 ceph-mon[37002]: pgmap v491: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:46:56.827 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T23:46:58.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:57 smithi099 ceph-mon[31715]: pgmap v492: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:58.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:57 smithi149 ceph-mon[36083]: pgmap v492: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:58.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:57 smithi174 ceph-mon[37002]: pgmap v492: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:46:59.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003603286, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000816992, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:59.265 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 109, 2024-03-23T23:46:59.266 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 106, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 445152, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 62960, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.034342232, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.012876984, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.022523933, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.003739414, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 105, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007802616, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000074310 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.267 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 445152, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 62960 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 82, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3184, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 92048, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 3, 2024-03-23T23:46:59.268 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.015770449, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.011052292, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.003239678, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 81, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007525067, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000092902 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:59.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 92048 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:46:59.270 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:46:59.271 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-23T23:46:59.272 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148863, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 28, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-03-23T23:46:59.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-23T23:46:59.274 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 110, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 429803, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 110, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 110, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 429803, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-23T23:46:59.276 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-23T23:47:00.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:46:59 smithi099 ceph-mon[31715]: pgmap v493: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:00.289 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-23T23:47:00.289 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-23T23:47:00.289 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-23T23:47:00.289 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-23T23:47:00.289 DEBUG:tasks.check_counter:Getting stats from b 2024-03-23T23:47:00.290 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-23T23:47:00.318 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:46:59 smithi149 ceph-mon[36083]: pgmap v493: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:00.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:46:59 smithi174 ceph-mon[37002]: pgmap v493: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:00.542 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:47:02.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:01 smithi149 ceph-mon[36083]: pgmap v494: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:02.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:01 smithi099 ceph-mon[31715]: pgmap v494: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:02.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:01 smithi174 ceph-mon[37002]: pgmap v494: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:02.561 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 4955, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 5375, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1269668, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 10198588, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 10, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.322403457, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.118758511, 2024-03-23T23:47:02.562 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.088819588, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.091911387, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5379, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.751309615, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000139674 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2294, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000407554, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000177 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-23T23:47:02.563 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 4030, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 4229, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1622879, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 5901633, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.250612645, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.089284926, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.079066969, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.074116625, 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:02.564 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4229, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.690820904, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000163353 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1142, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000209066, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000183 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 496400, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 70640 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 8010, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 14088, 2024-03-23T23:47:02.565 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 2753227, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 6591875, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 76, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.514359700, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.234078126, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.284409318, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.036867057, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 14080, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.604397285, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000042925 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:02.566 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 14176, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002545440, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000179 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 62944, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 328736 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-23T23:47:02.567 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7567, 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.865165845, 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000510792 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:02.568 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5743, 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.186325367, 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000032443 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-23T23:47:02.569 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 2489, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 2489, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2489, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 6.952990552, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002793487 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 57, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-23T23:47:02.570 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 223, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 5, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 218, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 215, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 1, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 1, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 2, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 4976, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 2497, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-23T23:47:02.571 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 64, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-23T23:47:02.572 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 64, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 42, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 5, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 37, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 22, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 1244, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 4852, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 3666, 2024-03-23T23:47:02.573 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 1212, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 14, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 2448 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 24, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 38, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:02.574 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 1212, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 1212, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 5, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:02.575 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 6136, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 3433, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 3433, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 2703, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 2722, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 18, 2024-03-23T23:47:02.576 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 14, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 14, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 4, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 1, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 4, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 11585471, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 16875023, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3670, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 5.734486074, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001562530 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.577 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 23, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 1240, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 1217, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 18, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 24, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 6, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 223, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1243, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 1020, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 1, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 1224, 2024-03-23T23:47:02.578 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 1223, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 56764, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 231804 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 5127, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 2489, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 10, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1210, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.611564383, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002984763 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.579 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 17, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003458341, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000203431 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.580 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 22, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005236860, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000238039 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.581 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.582 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-23T23:47:02.583 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.583 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.583 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.583 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 12, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.033805427, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002817118 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003056406, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000509401 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.004562315, 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002281157 2024-03-23T23:47:02.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.042428622, 2024-03-23T23:47:02.586 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.007071437 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000822915, 2024-03-23T23:47:02.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000822915 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.014902051, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002128864 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1206, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.233153232, 2024-03-23T23:47:02.589 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002680889 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-23T23:47:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 2056, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 2056, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 197, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:02.591 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:02.592 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 291737, 2024-03-23T23:47:02.593 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 121, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 528, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 6, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 162, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:02.594 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 243298, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2350, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 8373, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 13258636, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 37, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 8373, 2024-03-23T23:47:02.595 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8373, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 42.903118536, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005123984 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8373, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 9261 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 8373, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 106, 2024-03-23T23:47:02.596 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 8267, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 36, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 370, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 37, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 3702, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 60, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 4014, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:02.597 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 69, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 740, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-23T23:47:02.598 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 87, 2024-03-23T23:47:02.599 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 3, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 209, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 24 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-23T23:47:02.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 9, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 4, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 3498, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 1212, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:02.601 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 16991, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 4098278, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 16991, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 16991, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4098278, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.602 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 8373, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 13381048, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 8373, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 13381048, 2024-03-23T23:47:02.603 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 8373, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 8373, 2024-03-23T23:47:02.604 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 8373, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 8373, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1212, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 122412, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:02.605 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1212, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 32, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 122412, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x558fef3d0500": { 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:02.606 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 6136, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 12680719, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 6136, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 3670, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 12680719, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:02.607 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-23T23:47:03.409 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-23T23:47:03.409 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-23T23:47:03.409 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=24 2024-03-23T23:47:03.409 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-23T23:47:03.409 DEBUG:tasks.check_counter:Getting stats from e 2024-03-23T23:47:03.410 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-23T23:47:03.752 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:47:04.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:03 smithi149 ceph-mon[36083]: pgmap v495: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:04.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:03 smithi099 ceph-mon[31715]: pgmap v495: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:04.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:03 smithi174 ceph-mon[37002]: pgmap v495: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:05.612 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 118, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 114, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 468578, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 64468, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.014492518, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.004030998, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.010122149, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.001821153, 2024-03-23T23:47:05.613 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 113, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005163900, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000045698 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 467744, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 64144 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:05.614 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 276, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 354, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 69786, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 375465, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.021607286, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.007244968, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.005697191, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.006677956, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 352, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.014173265, 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000040264 2024-03-23T23:47:05.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 289600 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 701, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 703, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 210859, 2024-03-23T23:47:05.616 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 222976, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.042681701, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.014190459, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.011483230, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.013365556, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 701, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.018103802, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000025825 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1669, 2024-03-23T23:47:05.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.029641284, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000017759 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-23T23:47:05.619 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-23T23:47:05.620 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-23T23:47:05.621 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-03-23T23:47:05.622 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:05.623 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-23T23:47:05.624 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-23T23:47:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 5, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-23T23:47:05.626 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195730, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 4195424, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 5 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 3, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 3, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 0, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 1, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 1, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 0, 2024-03-23T23:47:05.627 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 46076, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.628 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.629 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-23T23:47:05.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.631 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.633 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.634 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-23T23:47:05.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-23T23:47:05.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-23T23:47:05.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-23T23:47:05.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-03-23T23:47:05.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1787, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:05.640 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:05.641 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 189217, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 48, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 162, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:05.643 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 8320, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 13, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 972, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 972, 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-23T23:47:05.644 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 972, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.576988154, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000593609 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 972, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 973 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 972, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 972, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 616, 2024-03-23T23:47:05.645 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 354, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-23T23:47:05.647 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 87, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 5, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 5, 2024-03-23T23:47:05.648 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-23T23:47:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1091, 2024-03-23T23:47:05.650 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 641992, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1091, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1091, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 641992, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:05.651 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 972, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1431, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 972, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1431, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.652 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 972, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 972, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 972, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 972, 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:05.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:05.654 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x5615ff60a6e0": { 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:05.655 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:05.656 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-23T23:47:06.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:05 smithi099 ceph-mon[31715]: pgmap v496: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T23:47:06.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:05 smithi149 ceph-mon[36083]: pgmap v496: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T23:47:06.426 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:05 smithi174 ceph-mon[37002]: pgmap v496: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-03-23T23:47:06.551 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-23T23:47:06.552 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-23T23:47:06.552 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-23T23:47:06.552 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-23T23:47:06.552 DEBUG:tasks.check_counter:Getting stats from h 2024-03-23T23:47:06.552 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-23T23:47:06.800 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:47:08.293 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:07 smithi149 ceph-mon[36083]: pgmap v497: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-23T23:47:08.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:07 smithi099 ceph-mon[31715]: pgmap v497: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-23T23:47:08.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:07 smithi174 ceph-mon[37002]: pgmap v497: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-03-23T23:47:08.597 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 836, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 830, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 4877470, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 291196, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.058813528, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.019196595, 2024-03-23T23:47:08.598 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.022394407, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.015618888, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 830, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.021733700, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000026185 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 470384, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 63504 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.599 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 94, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 172, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 4316989, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 242168, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.011304723, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.003674685, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.004977295, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.001691782, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 170, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.009675784, 2024-03-23T23:47:08.600 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000056916 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 212528 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 307, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 308, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 4270552, 2024-03-23T23:47:08.601 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 97315, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.021570020, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.005376097, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.007231024, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.007217534, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 306, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.007637406, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000024958 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.602 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1857, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.039153960, 2024-03-23T23:47:08.603 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000021084 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-23T23:47:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 1235, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 365, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 870, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:08.605 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 713, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 2, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-23T23:47:08.606 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:08.607 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-23T23:47:08.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:08.609 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-23T23:47:08.610 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 6136, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 4, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 12, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 11585471, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 16875023, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 16873371, 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-23T23:47:08.611 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 6136 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 23, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 1235, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 1212, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 18, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 24, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 6, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 1235, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1242, 2024-03-23T23:47:08.612 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 7, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 51856, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.618 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:08.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.620 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.622 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-23T23:47:08.624 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1790, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:08.625 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:08.626 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 189257, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 49, 2024-03-23T23:47:08.627 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 162, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 942362, 2024-03-23T23:47:08.628 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 5229, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 1115, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 26, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 4, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 1115, 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-23T23:47:08.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1115, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.507968998, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001352438 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1115, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1116 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 1115, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1115, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 698, 2024-03-23T23:47:08.630 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 414, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 2, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-23T23:47:08.631 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-23T23:47:08.632 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 87, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 5, 2024-03-23T23:47:08.633 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 5, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-23T23:47:08.634 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1233, 2024-03-23T23:47:08.635 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 13345101, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1233, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1233, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 13345101, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:08.636 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1115, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 12680729, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1115, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 12680729, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1115, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1115, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1115, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1115, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:08.639 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55b82b6fa6e0": { 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:08.640 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:08.641 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-23T23:47:09.286 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-23T23:47:09.287 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-23T23:47:09.287 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-23T23:47:09.287 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-23T23:47:09.287 DEBUG:tasks.check_counter:Getting stats from k 2024-03-23T23:47:09.287 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-23T23:47:09.539 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T23:47:10.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:09 smithi099 ceph-mon[31715]: pgmap v498: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:47:10.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:09 smithi149 ceph-mon[36083]: pgmap v498: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:47:10.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:09 smithi174 ceph-mon[37002]: pgmap v498: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 355, 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 357, 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 119225, 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 112891, 2024-03-23T23:47:11.297 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.026262627, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.007270612, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.007518479, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.008186168, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 355, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.012700283, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000035775 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.298 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 562, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 643, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 146828, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 392639, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.033423384, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.012127746, 2024-03-23T23:47:11.299 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.008544513, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.010055038, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 642, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.019352776, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000030144 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 215456 2024-03-23T23:47:11.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 124, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 120, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 470180, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 67605, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.015390421, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.004405679, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.011500222, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.002030846, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 119, 2024-03-23T23:47:11.301 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.006750787, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000056729 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 468832, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 66640 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-23T23:47:11.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1622, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.029684203, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000018300 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.303 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:11.304 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-23T23:47:11.305 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-03-23T23:47:11.306 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:11.307 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:11.308 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-23T23:47:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 3, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-23T23:47:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195198, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 4194892, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 3 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 3, 2024-03-23T23:47:11.311 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 3, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 1, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 1, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 46960, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-23T23:47:11.312 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.317 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.318 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.319 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.320 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.321 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.322 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:11.323 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-23T23:47:11.324 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1796, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:11.325 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:11.326 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 197297, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 52, 2024-03-23T23:47:11.327 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 162, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 8320, 2024-03-23T23:47:11.328 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 13, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 915, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 915, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 915, 2024-03-23T23:47:11.329 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.506989590, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000554086 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 915, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 916 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 915, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 915, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 554, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-03-23T23:47:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 359, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:11.331 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-23T23:47:11.332 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 87, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 6, 2024-03-23T23:47:11.333 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 6, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-23T23:47:11.334 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:11.335 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1037, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 633697, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1037, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1037, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 633697, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.336 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 915, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 899, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 915, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 899, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.337 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 915, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 915, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 915, 2024-03-23T23:47:11.338 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 915, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:11.339 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x556e69f866e0": { 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-23T23:47:11.340 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-23T23:47:11.341 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-23T23:47:12.256 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:12 smithi149 ceph-mon[36083]: pgmap v499: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-23T23:47:12.257 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-23T23:47:12.257 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-23T23:47:12.258 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-23T23:47:12.258 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-23T23:47:12.258 DEBUG:tasks.check_counter:Getting stats from c 2024-03-23T23:47:12.258 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-23T23:47:12.283 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:12 smithi174 ceph-mon[37002]: pgmap v499: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-23T23:47:12.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:12 smithi099 ceph-mon[31715]: pgmap v499: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-03-23T23:47:12.524 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:47:14.218 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:14 smithi174 ceph-mon[37002]: pgmap v500: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:14.360 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 876, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 657, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 694002, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 314325, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 18, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 8, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.055430482, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.018587893, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.043940547, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.003120786, 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:14.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 656, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.032362997, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000049333 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1196, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000210579, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000176 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 476128, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 69248 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 40, 2024-03-23T23:47:14.362 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 118, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 18735, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 282925, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.009249441, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.003379939, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.005068275, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000184251, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 117, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.023122383, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000197627 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.363 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 43, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000007124, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000165 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 216528 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 37, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 39, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 16364, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 20323, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-03-23T23:47:14.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.004151666, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.000697589, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.002973717, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000452798, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 37, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.019878092, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000537245 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 34, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000005419, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000159 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 51, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001424804, 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000027937 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.366 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000007302, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000003651 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 1, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 1, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-03-23T23:47:14.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000095201, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000095201 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 18, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 2, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 16, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 13, 2024-03-23T23:47:14.368 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 8, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 1, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 1, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 3, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 8, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 5, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 3, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-03-23T23:47:14.369 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:14.370 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-03-23T23:47:14.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 13, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 8, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 9, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-03-23T23:47:14.372 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-03-23T23:47:14.373 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 3, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 3, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4195198, 2024-03-23T23:47:14.374 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003962599, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001981299 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 22, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 28, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 6, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 21, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 25, 2024-03-23T23:47:14.375 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 18, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 26, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 8, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 1, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 1, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 50884, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223612 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 1, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 1, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 3, 2024-03-23T23:47:14.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000095201, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000095201 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.381 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:14 smithi099 ceph-mon[31715]: pgmap v500: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.382 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:14.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:14.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.388 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-03-23T23:47:14.389 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 1, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 1, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 200, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:14.390 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:14.391 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:14.392 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 279623, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 101, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 162, 2024-03-23T23:47:14.393 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 88229, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 345, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-03-23T23:47:14.394 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 38, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 11317, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 31, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 38, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.261996440, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006894643 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 61 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 38, 2024-03-23T23:47:14.395 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 38, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 2, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 6, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-03-23T23:47:14.396 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-03-23T23:47:14.397 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-03-23T23:47:14.398 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 87, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 10, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 10, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 16, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 1 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:14.400 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 949, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 633315, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 949, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 949, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 633315, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.401 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 38, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 11317, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 38, 2024-03-23T23:47:14.402 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 11317, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:14.403 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 38, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 38, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 38, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 38, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:14.404 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55fdfc0fc500": { 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:14.405 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 3, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 894, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 3, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 894, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:14.406 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.407 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:14.407 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-23T23:47:14.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:14 smithi149 ceph-mon[36083]: pgmap v500: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-23T23:47:15.074 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-23T23:47:15.074 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-23T23:47:15.074 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-23T23:47:15.075 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=13 2024-03-23T23:47:15.075 DEBUG:tasks.check_counter:Getting stats from f 2024-03-23T23:47:15.075 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-23T23:47:15.378 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:47:16.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:16 smithi099 ceph-mon[31715]: pgmap v501: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:16.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:16 smithi149 ceph-mon[36083]: pgmap v501: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:16.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:16 smithi174 ceph-mon[37002]: pgmap v501: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:17.118 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-23T23:47:17.118 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:17.118 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1008, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1007, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 290299, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 319409, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.055391642, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.016690781, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.014518249, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.019821158, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1006, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.020149798, 2024-03-23T23:47:17.119 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000020029 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 86, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 876, 2024-03-23T23:47:17.120 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 295178, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.004512600, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.002598148, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.001309906, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000048180, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 84, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.006710732, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000079889 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.121 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 294480 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 120, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 116, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 468556, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 65710, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 3, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.014153232, 2024-03-23T23:47:17.122 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.004234509, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.010249613, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.001903382, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 115, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.004312283, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000037498 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 468288, 2024-03-23T23:47:17.123 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 65392 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1725, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.030651414, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000017768 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.124 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.125 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 0, 2024-03-23T23:47:17.126 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-03-23T23:47:17.127 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:17.128 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-03-23T23:47:17.129 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:17.130 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 5, 2024-03-23T23:47:17.131 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4195730, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 4195424, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.132 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 5 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 3, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 3, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 1, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 1, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-03-23T23:47:17.133 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 46600, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215420 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.134 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.135 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.136 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.137 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.138 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.139 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.140 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.141 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-03-23T23:47:17.142 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-03-23T23:47:17.143 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-03-23T23:47:17.144 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-03-23T23:47:17.145 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1815, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:17.146 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:17.147 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:17.148 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 181137, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 44, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 162, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:17.149 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 8320, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 13, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 1006, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-03-23T23:47:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 1006, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1006, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.417980784, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001409523 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1006, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1007 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 1006, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 1006, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-03-23T23:47:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 639, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 365, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-03-23T23:47:17.153 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-03-23T23:47:17.154 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 87, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 4, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 4, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-03-23T23:47:17.155 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:17.156 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1128, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 649066, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1128, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1128, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 649066, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.157 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 1006, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1431, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1006, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1431, 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:17.158 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 1006, 2024-03-23T23:47:17.159 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1006, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1006, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1006, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:17.160 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55cc82fc46e0": { 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:17.161 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:17.162 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-23T23:47:17.811 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-23T23:47:17.811 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-23T23:47:17.811 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-23T23:47:17.812 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-23T23:47:17.812 DEBUG:tasks.check_counter:Getting stats from i 2024-03-23T23:47:17.812 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-23T23:47:18.056 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:47:18.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:18 smithi174 ceph-mon[37002]: pgmap v502: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:47:18.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:18 smithi099 ceph-mon[31715]: pgmap v502: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:47:18.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:18 smithi149 ceph-mon[36083]: pgmap v502: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 49, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 51, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 17919, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 173387, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 23, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.009150260, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.001174277, 2024-03-23T23:47:19.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.005829945, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000347790, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 49, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.020403523, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000416398 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 46, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000006197, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000134 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.785 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 185, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 176, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 500144, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 86940, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.017906159, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.005185591, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.013440765, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.002204812, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 176, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.021107118, 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000119926 2024-03-23T23:47:19.786 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 32, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000005859, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000183 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 485600, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 70016 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 723, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 593, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 220401, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 330944, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-03-23T23:47:19.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.040671102, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.016218169, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.027891288, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.001391955, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 591, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.026044022, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000044067 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1207, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000180688, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000149 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 217680 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001324887, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000030111 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.789 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000010841, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000005420 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 18, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 2, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 16, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 13, 2024-03-23T23:47:19.791 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 8, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 3, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 8, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 5, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 3, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-03-23T23:47:19.792 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:19.793 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 13, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 8, 2024-03-23T23:47:19.794 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 9, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:19.795 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 2, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 2, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-03-23T23:47:19.796 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4194892, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001663118, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001663118 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-03-23T23:47:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 22, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 28, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 6, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 21, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 25, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 18, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 26, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 8, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51032, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223612 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.799 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.800 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.801 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.802 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-03-23T23:47:19.803 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-03-23T23:47:19.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.806 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.809 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-03-23T23:47:19.810 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:19.811 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:19.812 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:19.813 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 230215, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 77, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 162, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:19.814 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 87853, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 342, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 35, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 10710, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 35, 2024-03-23T23:47:19.815 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 35, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.263996414, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007542754 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 35, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 57 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 35, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 35, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-03-23T23:47:19.816 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 1, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 5, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-03-23T23:47:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-03-23T23:47:19.818 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 87, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 8, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 8, 2024-03-23T23:47:19.819 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 14, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 1 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-03-23T23:47:19.820 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 953, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 642128, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:19.821 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 953, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 953, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 642128, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:19.822 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 35, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 10710, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 35, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 10710, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.823 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 35, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 35, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 35, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 35, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.824 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.825 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x561abf94a500": { 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 2, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 588, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 2, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1, 2024-03-23T23:47:19.826 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 588, 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:19.827 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-23T23:47:20.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:20 smithi174 ceph-mon[37002]: pgmap v503: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:20.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:20 smithi099 ceph-mon[31715]: pgmap v503: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:20.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:20 smithi149 ceph-mon[36083]: pgmap v503: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-23T23:47:20.534 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-23T23:47:20.534 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-23T23:47:20.534 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-23T23:47:20.534 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=13 2024-03-23T23:47:20.534 DEBUG:tasks.check_counter:Getting stats from l 2024-03-23T23:47:20.535 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-23T23:47:20.777 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T23:47:22.268 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:22 smithi174 ceph-mon[37002]: pgmap v504: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:22.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:22 smithi099 ceph-mon[31715]: pgmap v504: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:22.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:22 smithi149 ceph-mon[36083]: pgmap v504: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-23T23:47:22.555 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 489, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 485, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 592288, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 184582, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 1, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.042093612, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.012632191, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.018249521, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.011145954, 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:22.556 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 484, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.014611072, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000030188 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 469648, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 68512 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 8, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 92, 2024-03-23T23:47:22.557 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 4574, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 220206, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.004857019, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.002761685, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.001453235, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000067120, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 91, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.006705842, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000073690 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:22.558 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 219184 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 368, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 370, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 97013, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 117025, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-03-23T23:47:22.559 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.023132759, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.008287659, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.006290839, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.006624083, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 368, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.009712600, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000026392 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T23:47:22.560 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1465, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.029668483, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000020251 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.561 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.562 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-03-23T23:47:22.563 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-03-23T23:47:22.564 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-03-23T23:47:22.565 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-03-23T23:47:22.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-03-23T23:47:22.567 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 2, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-03-23T23:47:22.568 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4194892, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 4194352, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 2 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-03-23T23:47:22.569 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 3, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 3, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 1, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 1, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 46960, 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215420 2024-03-23T23:47:22.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-03-23T23:47:22.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-03-23T23:47:22.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.580 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-03-23T23:47:22.581 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-03-23T23:47:22.582 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T23:47:22.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-03-23T23:47:22.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 189195, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 48, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-03-23T23:47:22.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7720, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 162, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 8320, 2024-03-23T23:47:22.586 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 13, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 736, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 736, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 736, 2024-03-23T23:47:22.587 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.310995779, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000422548 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 736, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 737 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 736, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 736, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 365, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 369, 2024-03-23T23:47:22.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-03-23T23:47:22.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-03-23T23:47:22.590 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 87, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 5, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 5, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-03-23T23:47:22.591 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-03-23T23:47:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 861, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 607023, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 861, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:22.593 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 861, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 607023, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:22.594 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 736, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 593, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 736, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 593, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-03-23T23:47:22.595 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 736, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 736, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 736, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 736, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.596 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:22.597 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x562f1e5746e0": { 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-03-23T23:47:22.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-23T23:47:22.599 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-23T23:47:23.344 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-23T23:47:23.344 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-23T23:47:23.344 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-23T23:47:23.344 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-23T23:47:23.344 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-23T23:47:23.355 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T23:47:23.355 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:47:23.355 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T23:47:23.407 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-23T23:47:23.407 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T23:47:23.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:47:23.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 -- 172.21.15.99:0/8412648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481052a0 msgr2=0x7eff4810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:23.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 --2- 172.21.15.99:0/8412648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481052a0 0x7eff4810bb30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7eff40009ca0 tx=0x7eff4002c100 comp rx=0 tx=0).stop 2024-03-23T23:47:23.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 -- 172.21.15.99:0/8412648 shutdown_connections 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 --2- 172.21.15.99:0/8412648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481052a0 0x7eff4810bb30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 --2- 172.21.15.99:0/8412648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481048e0 0x7eff48104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 --2- 172.21.15.99:0/8412648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481036f0 0x7eff48103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 -- 172.21.15.99:0/8412648 >> 172.21.15.99:0/8412648 conn(0x7eff480fec80 msgr2=0x7eff481010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 -- 172.21.15.99:0/8412648 shutdown_connections 2024-03-23T23:47:23.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.710+0000 7eff4d200640 1 -- 172.21.15.99:0/8412648 wait complete. 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.711+0000 7eff4d200640 1 Processor -- start 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.711+0000 7eff4d200640 1 -- start start 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.711+0000 7eff4d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.711+0000 7eff4d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 0x7eff48199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff4d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481052a0 0x7eff481a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:23.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff4d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810f350 con 0x7eff481048e0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff4d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810f1d0 con 0x7eff481052a0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff4d200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4810f4d0 con 0x7eff481036f0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff46a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 0x7eff48199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50650/0 (socket says 172.21.15.99:50650) 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 -- 172.21.15.99:0/51709026 learned_addr learned my addr 172.21.15.99:0/51709026 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47e00640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481052a0 0x7eff481a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 -- 172.21.15.99:0/51709026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481052a0 msgr2=0x7eff481a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481052a0 0x7eff481a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 -- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 msgr2=0x7eff48199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 0x7eff48199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff47400640 1 -- 172.21.15.99:0/51709026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff481a15e0 con 0x7eff481036f0 2024-03-23T23:47:23.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.712+0000 7eff46a00640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 0x7eff48199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:23.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff47400640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e9c0 tx=0x7eff3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:23.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3800cd80 con 0x7eff481036f0 2024-03-23T23:47:23.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff380035f0 con 0x7eff481036f0 2024-03-23T23:47:23.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff481a18d0 con 0x7eff481036f0 2024-03-23T23:47:23.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38010690 con 0x7eff481036f0 2024-03-23T23:47:23.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.713+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff481a1e70 con 0x7eff481036f0 2024-03-23T23:47:23.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.714+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff48068800 con 0x7eff481036f0 2024-03-23T23:47:23.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.716+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7eff380037a0 con 0x7eff481036f0 2024-03-23T23:47:23.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.718+0000 7eff3fe00640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 0x7eff2c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:23.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.718+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7eff380cedf0 con 0x7eff481036f0 2024-03-23T23:47:23.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.718+0000 7eff46a00640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 0x7eff2c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:23.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.719+0000 7eff46a00640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 0x7eff2c0a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff4819aed0 tx=0x7eff34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:23.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.719+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff3809be10 con 0x7eff481036f0 2024-03-23T23:47:23.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.931+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff4802d0f0 con 0x7eff481036f0 2024-03-23T23:47:23.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.934+0000 7eff3fe00640 1 -- 172.21.15.99:0/51709026 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7eff3809b710 con 0x7eff481036f0 2024-03-23T23:47:23.934 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:47:23.934 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 msgr2=0x7eff2c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 0x7eff2c0a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff4819aed0 tx=0x7eff34005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 msgr2=0x7eff481999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e9c0 tx=0x7eff3800ee80 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 shutdown_connections 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7eff2c0a2880 0x7eff2c0a4d40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481052a0 0x7eff481a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff481048e0 0x7eff48199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 --2- 172.21.15.99:0/51709026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff481036f0 0x7eff481999c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.936+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 >> 172.21.15.99:0/51709026 conn(0x7eff480fec80 msgr2=0x7eff48107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.937+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 shutdown_connections 2024-03-23T23:47:23.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:23.937+0000 7eff4d200640 1 -- 172.21.15.99:0/51709026 wait complete. 2024-03-23T23:47:23.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T23:35:55.293260+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T23:41:08.102728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T23:41:09.219534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T23:47:23.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T23:47:24.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 -- 172.21.15.99:0/3455350211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee80ffe50 msgr2=0x7faee8100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 --2- 172.21.15.99:0/3455350211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee80ffe50 0x7faee8100270 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faed8009860 tx=0x7faed802bb70 comp rx=0 tx=0).stop 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 -- 172.21.15.99:0/3455350211 shutdown_connections 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 --2- 172.21.15.99:0/3455350211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8101150 0x7faee81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 --2- 172.21.15.99:0/3455350211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 --2- 172.21.15.99:0/3455350211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee80ffe50 0x7faee8100270 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 -- 172.21.15.99:0/3455350211 >> 172.21.15.99:0/3455350211 conn(0x7faee80fba10 msgr2=0x7faee80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 -- 172.21.15.99:0/3455350211 shutdown_connections 2024-03-23T23:47:24.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.161+0000 7faee7e00640 1 -- 172.21.15.99:0/3455350211 wait complete. 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.162+0000 7faee7e00640 1 Processor -- start 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 -- start start 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 0x7faee8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee8101150 0x7faee81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f3c0 con 0x7faee80ffe50 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f240 con 0x7faee8101150 2024-03-23T23:47:24.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee810f540 con 0x7faee81007b0 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee6a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 0x7faee8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee6000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee6000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50670/0 (socket says 172.21.15.99:50670) 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee6000640 1 -- 172.21.15.99:0/3032391190 learned_addr learned my addr 172.21.15.99:0/3032391190 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.163+0000 7faee7400640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee8101150 0x7faee81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 -- 172.21.15.99:0/3032391190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee8101150 msgr2=0x7faee81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee8101150 0x7faee81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 -- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 msgr2=0x7faee8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 0x7faee8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 -- 172.21.15.99:0/3032391190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee81a1680 con 0x7faee81007b0 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6a00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 0x7faee8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:24.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee6000640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faed400e9c0 tx=0x7faed400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:24.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400ccc0 con 0x7faee81007b0 2024-03-23T23:47:24.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed4005500 con 0x7faee81007b0 2024-03-23T23:47:24.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee81a1970 con 0x7faee81007b0 2024-03-23T23:47:24.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.164+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4010430 con 0x7faee81007b0 2024-03-23T23:47:24.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.165+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee81a1ec0 con 0x7faee81007b0 2024-03-23T23:47:24.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.166+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee810ad90 con 0x7faee81007b0 2024-03-23T23:47:24.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.168+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faed40046c0 con 0x7faee81007b0 2024-03-23T23:47:24.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.170+0000 7faeeca00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 0x7faecc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.171+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faed40cedd0 con 0x7faee81007b0 2024-03-23T23:47:24.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.171+0000 7faee6a00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 0x7faecc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.171+0000 7faee6a00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 0x7faecc0a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faed8009830 tx=0x7faed802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:24.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.173+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faed409bdf0 con 0x7faee81007b0 2024-03-23T23:47:24.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:24 smithi174 ceph-mon[37002]: pgmap v505: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:47:24.354 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:24 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/51709026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:24 smithi099 ceph-mon[31715]: pgmap v505: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:47:24.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:24 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/51709026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:24.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.392+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faee80622a0 con 0x7faee81007b0 2024-03-23T23:47:24.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.393+0000 7faeeca00640 1 -- 172.21.15.99:0/3032391190 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7faed409b6f0 con 0x7faee81007b0 2024-03-23T23:47:24.393 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-23T23:47:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:24 smithi149 ceph-mon[36083]: pgmap v505: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-23T23:47:24.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:24 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/51709026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 msgr2=0x7faecc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 0x7faecc0a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faed8009830 tx=0x7faed802e040 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 msgr2=0x7faee8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faed400e9c0 tx=0x7faed400ee80 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 shutdown_connections 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7faecc0a2800 0x7faecc0a4cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faee8101150 0x7faee81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faee81007b0 0x7faee8199fa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 --2- 172.21.15.99:0/3032391190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee80ffe50 0x7faee8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.396+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 >> 172.21.15.99:0/3032391190 conn(0x7faee80fba10 msgr2=0x7faee80fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.397+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 shutdown_connections 2024-03-23T23:47:24.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.397+0000 7faee7e00640 1 -- 172.21.15.99:0/3032391190 wait complete. 2024-03-23T23:47:24.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 -- 172.21.15.99:0/3641400778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4103150 msgr2=0x7f51b41099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 --2- 172.21.15.99:0/3641400778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4103150 0x7f51b41099e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009be0 tx=0x7f51ac02bef0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 -- 172.21.15.99:0/3641400778 shutdown_connections 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 --2- 172.21.15.99:0/3641400778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4103150 0x7f51b41099e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 --2- 172.21.15.99:0/3641400778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b4102790 0x7f51b4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 --2- 172.21.15.99:0/3641400778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b41015a0 0x7f51b41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.635+0000 7f51b9000640 1 -- 172.21.15.99:0/3641400778 >> 172.21.15.99:0/3641400778 conn(0x7f51b40fcb10 msgr2=0x7f51b40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.636+0000 7f51b9000640 1 -- 172.21.15.99:0/3641400778 shutdown_connections 2024-03-23T23:47:24.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.636+0000 7f51b9000640 1 -- 172.21.15.99:0/3641400778 wait complete. 2024-03-23T23:47:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.637+0000 7f51b9000640 1 Processor -- start 2024-03-23T23:47:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.637+0000 7f51b9000640 1 -- start start 2024-03-23T23:47:24.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.637+0000 7f51b9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 0x7f51b4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b9000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b4103150 0x7f51b419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b9000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b410d250 con 0x7f51b4103150 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b9000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b410d0d0 con 0x7f51b4102790 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b9000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b410d3d0 con 0x7f51b41015a0 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50684/0 (socket says 172.21.15.99:50684) 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 -- 172.21.15.99:0/3796439344 learned_addr learned my addr 172.21.15.99:0/3796439344 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b2a00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 0x7f51b4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3e00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b4103150 0x7f51b419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 -- 172.21.15.99:0/3796439344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 msgr2=0x7f51b4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 0x7f51b4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 -- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b4103150 msgr2=0x7f51b419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b4103150 0x7f51b419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b3400640 1 -- 172.21.15.99:0/3796439344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b419f430 con 0x7f51b41015a0 2024-03-23T23:47:24.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.638+0000 7f51b2a00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 0x7f51b4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:24.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51b3400640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f51a400e9c0 tx=0x7f51a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a400cd80 con 0x7f51b41015a0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51a40035f0 con 0x7f51b41015a0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b419f720 con 0x7f51b41015a0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4010690 con 0x7f51b41015a0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.639+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b419fcc0 con 0x7f51b41015a0 2024-03-23T23:47:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.641+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4108860 con 0x7f51b41015a0 2024-03-23T23:47:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.644+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f51a40037a0 con 0x7f51b41015a0 2024-03-23T23:47:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.645+0000 7f51abe00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 0x7f51980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:24.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.645+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f51a40cee10 con 0x7f51b41015a0 2024-03-23T23:47:24.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.646+0000 7f51b2a00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 0x7f51980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:24.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.646+0000 7f51b2a00640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 0x7f51980a4d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096c0 tx=0x7f51a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:24.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.649+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f51a40d2030 con 0x7f51b41015a0 2024-03-23T23:47:24.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.893+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51b40622a0 con 0x7f51b41015a0 2024-03-23T23:47:24.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.893+0000 7f51abe00640 1 -- 172.21.15.99:0/3796439344 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51a409be30 con 0x7f51b41015a0 2024-03-23T23:47:24.894 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.896+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 msgr2=0x7f51980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.896+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 0x7f51980a4d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51a00096c0 tx=0x7f51a0009260 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.896+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 msgr2=0x7f51b4197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.896+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f51a400e9c0 tx=0x7f51a400ee80 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 shutdown_connections 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f51980a2880 0x7f51980a4d40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b4103150 0x7f51b419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4102790 0x7f51b4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 --2- 172.21.15.99:0/3796439344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51b41015a0 0x7f51b4197810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:24.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 >> 172.21.15.99:0/3796439344 conn(0x7f51b40fcb10 msgr2=0x7f51b4105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 shutdown_connections 2024-03-23T23:47:24.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:24.897+0000 7f51b9000640 1 -- 172.21.15.99:0/3796439344 wait complete. 2024-03-23T23:47:24.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 -- 172.21.15.99:0/3222906499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141028a0 msgr2=0x7fd714102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 --2- 172.21.15.99:0/3222906499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141028a0 0x7fd714102d20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd700009900 tx=0x7fd70002bc30 comp rx=0 tx=0).stop 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 -- 172.21.15.99:0/3222906499 shutdown_connections 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 --2- 172.21.15.99:0/3222906499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 0x7fd714109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 --2- 172.21.15.99:0/3222906499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141028a0 0x7fd714102d20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 --2- 172.21.15.99:0/3222906499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141016b0 0x7fd714101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.125+0000 7fd71aa00640 1 -- 172.21.15.99:0/3222906499 >> 172.21.15.99:0/3222906499 conn(0x7fd7140fcc40 msgr2=0x7fd7140ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:25.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.126+0000 7fd71aa00640 1 -- 172.21.15.99:0/3222906499 shutdown_connections 2024-03-23T23:47:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.126+0000 7fd71aa00640 1 -- 172.21.15.99:0/3222906499 wait complete. 2024-03-23T23:47:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.127+0000 7fd71aa00640 1 Processor -- start 2024-03-23T23:47:25.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.127+0000 7fd71aa00640 1 -- start start 2024-03-23T23:47:25.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 0x7fd7141a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71410d360 con 0x7fd714103260 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71410d1e0 con 0x7fd7141028a0 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd71aa00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71410d4e0 con 0x7fd7141016b0 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50710/0 (socket says 172.21.15.99:50710) 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd718a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 0x7fd7141a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713e00640 1 -- 172.21.15.99:0/102498538 learned_addr learned my addr 172.21.15.99:0/102498538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:32852/0 (socket says 172.21.15.99:32852) 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 -- 172.21.15.99:0/102498538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 msgr2=0x7fd714199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 -- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 msgr2=0x7fd7141a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 0x7fd7141a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713400640 1 -- 172.21.15.99:0/102498538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7141a17c0 con 0x7fd7141028a0 2024-03-23T23:47:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.128+0000 7fd713e00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd713400640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd700036040 tx=0x7fd70002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:25.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700043020 con 0x7fd7141028a0 2024-03-23T23:47:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7000054e0 con 0x7fd7141028a0 2024-03-23T23:47:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7141a1a50 con 0x7fd7141028a0 2024-03-23T23:47:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700003c80 con 0x7fd7141028a0 2024-03-23T23:47:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.129+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7141a1fc0 con 0x7fd7141028a0 2024-03-23T23:47:25.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.130+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7140688e0 con 0x7fd7141028a0 2024-03-23T23:47:25.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.132+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd700046020 con 0x7fd7141028a0 2024-03-23T23:47:25.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.133+0000 7fd710c00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 0x7fd6f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.133+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd7000f0ae0 con 0x7fd7141028a0 2024-03-23T23:47:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.134+0000 7fd713e00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 0x7fd6f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.134+0000 7fd713e00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 0x7fd6f80a4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7040095f0 tx=0x7fd704005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:25.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.137+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd70004d020 con 0x7fd7141028a0 2024-03-23T23:47:25.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3032391190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:25 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3796439344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.378+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7141a2930 con 0x7fd7141028a0 2024-03-23T23:47:25.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.378+0000 7fd710c00640 1 -- 172.21.15.99:0/102498538 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7000bdb00 con 0x7fd7141028a0 2024-03-23T23:47:25.378 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 msgr2=0x7fd6f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 0x7fd6f80a4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd7040095f0 tx=0x7fd704005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 msgr2=0x7fd71419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd700036040 tx=0x7fd70002c6c0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 shutdown_connections 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd6f80a2880 0x7fd6f80a4d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd714103260 0x7fd7141a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7141028a0 0x7fd71419a0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 --2- 172.21.15.99:0/102498538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7141016b0 0x7fd714199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 >> 172.21.15.99:0/102498538 conn(0x7fd7140fcc40 msgr2=0x7fd7140738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 shutdown_connections 2024-03-23T23:47:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.382+0000 7fd71aa00640 1 -- 172.21.15.99:0/102498538 wait complete. 2024-03-23T23:47:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3032391190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:25 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3796439344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.403 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T23:47:25.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3032391190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:25 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3796439344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T23:47:25.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.594+0000 7f74cd400640 1 -- 172.21.15.99:0/2590860520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 msgr2=0x7f74c8073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.594+0000 7f74cd400640 1 --2- 172.21.15.99:0/2590860520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8073ad0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b802bcd0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 -- 172.21.15.99:0/2590860520 shutdown_connections 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 --2- 172.21.15.99:0/2590860520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c8109430 0x7f74c810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 --2- 172.21.15.99:0/2590860520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c80740a0 0x7f74c8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 --2- 172.21.15.99:0/2590860520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8073ad0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 -- 172.21.15.99:0/2590860520 >> 172.21.15.99:0/2590860520 conn(0x7f74c80fcb10 msgr2=0x7f74c80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 -- 172.21.15.99:0/2590860520 shutdown_connections 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.595+0000 7f74cd400640 1 -- 172.21.15.99:0/2590860520 wait complete. 2024-03-23T23:47:25.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 Processor -- start 2024-03-23T23:47:25.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 -- start start 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c8109430 0x7f74c81a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c810f460 con 0x7f74c8109430 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c810f2e0 con 0x7f74c80740a0 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cd400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c810f5e0 con 0x7f74c8075680 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74c7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:32868/0 (socket says 172.21.15.99:32868) 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74c7e00640 1 -- 172.21.15.99:0/1428791280 learned_addr learned my addr 172.21.15.99:0/1428791280 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:25.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74c7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50718/0 (socket says 172.21.15.99:50718) 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.596+0000 7f74cca00640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c8109430 0x7f74c81a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 -- 172.21.15.99:0/1428791280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 msgr2=0x7f74c8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 -- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c8109430 msgr2=0x7f74c81a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c8109430 0x7f74c81a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 -- 172.21.15.99:0/1428791280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c81a1660 con 0x7f74c80740a0 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c7e00640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74b8000c00 tx=0x7f74b802c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8043020 con 0x7f74c80740a0 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74b80054e0 con 0x7f74c80740a0 2024-03-23T23:47:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c81a18f0 con 0x7f74c80740a0 2024-03-23T23:47:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8003c20 con 0x7f74c80740a0 2024-03-23T23:47:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.597+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81a1e90 con 0x7f74c80740a0 2024-03-23T23:47:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.598+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c81081f0 con 0x7f74c80740a0 2024-03-23T23:47:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.600+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f74b8046020 con 0x7f74c80740a0 2024-03-23T23:47:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.602+0000 7f74c4c00640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 0x7f74ac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.603+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74b80f0ac0 con 0x7f74c80740a0 2024-03-23T23:47:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.603+0000 7f74c7400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 0x7f74ac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.603+0000 7f74c7400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 0x7f74ac0a4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f74c819af50 tx=0x7f74b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:25.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.605+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74b804d020 con 0x7f74c80740a0 2024-03-23T23:47:25.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.819+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74c80622a0 con 0x7f74c80740a0 2024-03-23T23:47:25.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.819+0000 7f74c4c00640 1 -- 172.21.15.99:0/1428791280 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7f74b80bdae0 con 0x7f74c80740a0 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/3702331558 2024-03-24T23:33:06.585425+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/2749486918 2024-03-24T23:33:06.585425+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1079793222 2024-03-24T23:32:29.874809+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2100135742 2024-03-24T23:32:29.874809+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/3938208331 2024-03-24T23:32:29.874809+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2517072551 2024-03-24T23:33:06.585425+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/2749486918 2024-03-24T23:33:06.585425+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/686293385 2024-03-24T23:33:06.585425+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/3938208331 2024-03-24T23:32:29.874809+0000 2024-03-23T23:47:25.820 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2402511247 2024-03-24T23:32:29.874809+0000 2024-03-23T23:47:25.821 INFO:teuthology.orchestra.run.smithi099.stderr:listed 10 entries 2024-03-23T23:47:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 msgr2=0x7f74ac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 0x7f74ac0a4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f74c819af50 tx=0x7f74b4005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 msgr2=0x7f74c8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74b8000c00 tx=0x7f74b802c6f0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 shutdown_connections 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f74ac0a2880 0x7f74ac0a4d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74c8109430 0x7f74c81a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c8075680 0x7f74c8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 --2- 172.21.15.99:0/1428791280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74c80740a0 0x7f74c8199a40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 >> 172.21.15.99:0/1428791280 conn(0x7f74c80fcb10 msgr2=0x7f74c81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 shutdown_connections 2024-03-23T23:47:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:25.822+0000 7f74cd400640 1 -- 172.21.15.99:0/1428791280 wait complete. 2024-03-23T23:47:25.842 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T23:47:25.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:47:25.843 DEBUG:teuthology.orchestra.run.smithi099:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-23T23:47:25.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:47:25.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:47:25.945 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T23:47:26.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:47:26.018 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-23T23:47:26.060 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T23:47:26.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:47:26.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T23:47:26.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:26 smithi099 ceph-mon[31715]: pgmap v506: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:26.351 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/102498538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:47:26.351 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:26 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1428791280' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T23:47:26.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T23:47:26.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-23T23:47:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:26 smithi149 ceph-mon[36083]: pgmap v506: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/102498538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:47:26.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:26 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1428791280' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T23:47:26.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:26 smithi174 ceph-mon[37002]: pgmap v506: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-23T23:47:26.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/102498538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T23:47:26.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:26 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1428791280' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T23:47:26.514 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-23T23:47:26.611 INFO:tasks.cephadm:Teardown begin 2024-03-23T23:47:26.611 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T23:47:26.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:47:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 -- 172.21.15.99:0/43775112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c105080 msgr2=0x7f437c105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 --2- 172.21.15.99:0/43775112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c105080 0x7f437c105500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ca0 tx=0x7f437402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 -- 172.21.15.99:0/43775112 shutdown_connections 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 --2- 172.21.15.99:0/43775112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c105080 0x7f437c105500 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 --2- 172.21.15.99:0/43775112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c1026d0 0x7f437c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 --2- 172.21.15.99:0/43775112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 -- 172.21.15.99:0/43775112 >> 172.21.15.99:0/43775112 conn(0x7f437c0fb980 msgr2=0x7f437c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 -- 172.21.15.99:0/43775112 shutdown_connections 2024-03-23T23:47:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.819+0000 7f4381800640 1 -- 172.21.15.99:0/43775112 wait complete. 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.820+0000 7f4381800640 1 Processor -- start 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 -- start start 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 0x7f437c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c105080 0x7f437c1a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c113760 con 0x7f437c105080 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c1135e0 con 0x7f437c0ffd60 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f4381800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c1138e0 con 0x7f437c1026d0 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f437be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f437be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:32888/0 (socket says 172.21.15.99:32888) 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.821+0000 7f437be00640 1 -- 172.21.15.99:0/2050591464 learned_addr learned my addr 172.21.15.99:0/2050591464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f4380e00640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c105080 0x7f437c1a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437b400640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 0x7f437c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 -- 172.21.15.99:0/2050591464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 msgr2=0x7f437c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 0x7f437c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 -- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c105080 msgr2=0x7f437c1a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c105080 0x7f437c1a52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 -- 172.21.15.99:0/2050591464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c1a5960 con 0x7f437c0ffd60 2024-03-23T23:47:26.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437b400640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 0x7f437c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f437be00640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e9f0 tx=0x7f436c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.822+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00cd80 con 0x7f437c0ffd60 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.823+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f436c005500 con 0x7f437c0ffd60 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.823+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437c1a5c50 con 0x7f437c0ffd60 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.823+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c010690 con 0x7f437c0ffd60 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.823+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437c1a6150 con 0x7f437c0ffd60 2024-03-23T23:47:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.824+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437c0fd470 con 0x7f437c0ffd60 2024-03-23T23:47:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.827+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f436c0046c0 con 0x7f437c0ffd60 2024-03-23T23:47:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.829+0000 7f4378c00640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 0x7f43600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.829+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f436c0cef80 con 0x7f437c0ffd60 2024-03-23T23:47:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.829+0000 7f437b400640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 0x7f43600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.830+0000 7f437b400640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 0x7f43600a4cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f437c19f250 tx=0x7f4368005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:26.832+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f436c09bfa0 con 0x7f437c0ffd60 2024-03-23T23:47:27.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.104+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f437c0622d0 con 0x7f437c0ffd60 2024-03-23T23:47:27.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.105+0000 7f4378c00640 1 -- 172.21.15.99:0/2050591464 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f436c09b8a0 con 0x7f437c0ffd60 2024-03-23T23:47:27.105 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:47:27.106 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:47:27.106 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.108+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 msgr2=0x7f43600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.108+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 0x7f43600a4cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f437c19f250 tx=0x7f4368005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.108+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 msgr2=0x7f437c19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.108+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f436c00e9f0 tx=0x7f436c00edf0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 shutdown_connections 2024-03-23T23:47:27.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f43600a2800 0x7f43600a4cc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c105080 0x7f437c1a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1026d0 0x7f437c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 --2- 172.21.15.99:0/2050591464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f437c0ffd60 0x7f437c19dd40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 >> 172.21.15.99:0/2050591464 conn(0x7f437c0fb980 msgr2=0x7f437c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 shutdown_connections 2024-03-23T23:47:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.109+0000 7f4381800640 1 -- 172.21.15.99:0/2050591464 wait complete. 2024-03-23T23:47:27.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T23:47:27.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 -- 172.21.15.99:0/1453594775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 msgr2=0x7f44ac104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 --2- 172.21.15.99:0/1453594775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac104b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4498009900 tx=0x7f449802bc30 comp rx=0 tx=0).stop 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 -- 172.21.15.99:0/1453594775 shutdown_connections 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 --2- 172.21.15.99:0/1453594775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac105080 0x7f44ac105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 --2- 172.21.15.99:0/1453594775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac104b40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 --2- 172.21.15.99:0/1453594775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac0ffd60 0x7f44ac102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 -- 172.21.15.99:0/1453594775 >> 172.21.15.99:0/1453594775 conn(0x7f44ac0fb980 msgr2=0x7f44ac0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 -- 172.21.15.99:0/1453594775 shutdown_connections 2024-03-23T23:47:27.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.317+0000 7f44ab200640 1 -- 172.21.15.99:0/1453594775 wait complete. 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.318+0000 7f44ab200640 1 Processor -- start 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 -- start start 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 0x7f44ac1a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac113790 con 0x7f44ac105080 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac113610 con 0x7f44ac0ffd60 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44ab200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac113910 con 0x7f44ac1026d0 2024-03-23T23:47:27.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44a9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44a9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:32908/0 (socket says 172.21.15.99:32908) 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44a9e00640 1 -- 172.21.15.99:0/1704417382 learned_addr learned my addr 172.21.15.99:0/1704417382 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.319+0000 7f44a9400640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44aa800640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 0x7f44ac1a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 -- 172.21.15.99:0/1704417382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 msgr2=0x7f44ac19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 -- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 msgr2=0x7f44ac1a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 0x7f44ac1a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 -- 172.21.15.99:0/1704417382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac1a59d0 con 0x7f44ac0ffd60 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44aa800640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 0x7f44ac1a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a9e00640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f449c00e9f0 tx=0x7f449c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00cd80 con 0x7f44ac0ffd60 2024-03-23T23:47:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f449c010910 con 0x7f44ac0ffd60 2024-03-23T23:47:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c018990 con 0x7f44ac0ffd60 2024-03-23T23:47:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.320+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac1a5cc0 con 0x7f44ac0ffd60 2024-03-23T23:47:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.321+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac1a6190 con 0x7f44ac0ffd60 2024-03-23T23:47:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.322+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ac0fd470 con 0x7f44ac0ffd60 2024-03-23T23:47:27.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.323+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f449c010ac0 con 0x7f44ac0ffd60 2024-03-23T23:47:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.326+0000 7f44a2a00640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 0x7f44900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.326+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f449c0cee20 con 0x7f44ac0ffd60 2024-03-23T23:47:27.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.326+0000 7f44a9400640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 0x7f44900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.327+0000 7f44a9400640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 0x7f44900a4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f449802c240 tx=0x7f449802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:27.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.329+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f449c09be40 con 0x7f44ac0ffd60 2024-03-23T23:47:27.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.659+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44ac02d070 con 0x7f44ac0ffd60 2024-03-23T23:47:27.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.660+0000 7f44a2a00640 1 -- 172.21.15.99:0/1704417382 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11462 (secure 0 0 0) 0x7f449c09b740 con 0x7f44ac0ffd60 2024-03-23T23:47:27.660 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:47:27.660 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14391,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3136510370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3136510370},{"type":"v1","addr":"172.21.15.99:6835","nonce":3136510370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2449593351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2449593351},{"type":"v1","addr":"172.21.15.99:6837","nonce":2449593351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T23:41:09.728597+0000","modified":"2024-03-23T23:43:41.910870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24565,"mds_2":24557,"mds_3":24547,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2919310963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2919310963},{"type":"v1","addr":"172.21.15.149:6833","nonce":2919310963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/4085463188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4085463188},{"type":"v1","addr":"172.21.15.174:6833","nonce":4085463188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1177424904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1177424904},{"type":"v1","addr":"172.21.15.149:6835","nonce":1177424904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/731827456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":731827456},{"type":"v1","addr":"172.21.15.174:6835","nonce":731827456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3655333447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3655333447},{"type":"v1","addr":"172.21.15.149:6837","nonce":3655333447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24526":{"gid":24526,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":9,"addr":"172.21.15.174:6837/2363957261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2363957261},{"type":"v1","addr":"172.21.15.174:6837","nonce":2363957261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24547":{"gid":24547,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.174:6839/1404645838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":1404645838},{"type":"v1","addr":"172.21.15.174:6839","nonce":1404645838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24557":{"gid":24557,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/678226276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":678226276},{"type":"v1","addr":"172.21.15.99:6841","nonce":678226276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24565":{"gid":24565,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/4063961606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":4063961606},{"type":"v1","addr":"172.21.15.99:6839","nonce":4063961606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/2464794793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2464794793},{"type":"v1","addr":"172.21.15.149:6839","nonce":2464794793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24565,24569,24557,24547,24526]},"id":1}]} 2024-03-23T23:47:27.661 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-23T23:47:27.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.662+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 msgr2=0x7f44900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.662+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 0x7f44900a4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f449802c240 tx=0x7f449802df90 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.662+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 msgr2=0x7f44ac19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.662+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f449c00e9f0 tx=0x7f449c00eeb0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.662+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 shutdown_connections 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f44900a2880 0x7f44900a4d40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac105080 0x7f44ac1a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac1026d0 0x7f44ac19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 --2- 172.21.15.99:0/1704417382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44ac0ffd60 0x7f44ac19de70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 >> 172.21.15.99:0/1704417382 conn(0x7f44ac0fb980 msgr2=0x7f44ac100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:27.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 shutdown_connections 2024-03-23T23:47:27.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.663+0000 7f44ab200640 1 -- 172.21.15.99:0/1704417382 wait complete. 2024-03-23T23:47:27.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T23:47:27.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 -- 172.21.15.99:0/2346571713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e4103700 msgr2=0x7fb2e4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2346571713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e4103700 0x7fb2e4103b20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009860 tx=0x7fb2d402bb70 comp rx=0 tx=0).stop 2024-03-23T23:47:27.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 -- 172.21.15.99:0/2346571713 shutdown_connections 2024-03-23T23:47:27.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2346571713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2346571713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e41048f0 0x7fb2e4104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2346571713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e4103700 0x7fb2e4103b20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 -- 172.21.15.99:0/2346571713 >> 172.21.15.99:0/2346571713 conn(0x7fb2e40fec70 msgr2=0x7fb2e41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:27.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 -- 172.21.15.99:0/2346571713 shutdown_connections 2024-03-23T23:47:27.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.890+0000 7fb2e9800640 1 -- 172.21.15.99:0/2346571713 wait complete. 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 Processor -- start 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 -- start start 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 0x7fb2e4199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e41a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e410f350 con 0x7fb2e41052b0 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e410f1d0 con 0x7fb2e41048f0 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.892+0000 7fb2e9800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e410f4d0 con 0x7fb2e4103700 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 0x7fb2e4199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50776/0 (socket says 172.21.15.99:50776) 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 -- 172.21.15.99:0/2074208897 learned_addr learned my addr 172.21.15.99:0/2074208897 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e8e00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e41a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 -- 172.21.15.99:0/2074208897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 msgr2=0x7fb2e4199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 0x7fb2e4199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 -- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 msgr2=0x7fb2e41a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e41a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:27.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3e00640 1 -- 172.21.15.99:0/2074208897 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e41a1660 con 0x7fb2e4103700 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e3400640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 0x7fb2e4199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.893+0000 7fb2e8e00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e41a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e3e00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40349e0 tx=0x7fb2d4032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4041020 con 0x7fb2e4103700 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2d402c9f0 con 0x7fb2e4103700 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e41a18f0 con 0x7fb2e4103700 2024-03-23T23:47:27.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4039040 con 0x7fb2e4103700 2024-03-23T23:47:27.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.894+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e41a1de0 con 0x7fb2e4103700 2024-03-23T23:47:27.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.895+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e410aaf0 con 0x7fb2e4103700 2024-03-23T23:47:27.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.897+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb2d4044020 con 0x7fb2e4103700 2024-03-23T23:47:27.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.898+0000 7fb2e0c00640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 0x7fb2c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:27.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.899+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb2d40f0790 con 0x7fb2e4103700 2024-03-23T23:47:27.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.899+0000 7fb2e3400640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 0x7fb2c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:27.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.900+0000 7fb2e3400640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 0x7fb2c80a4cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e419af50 tx=0x7fb2d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:27.905+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2d404b020 con 0x7fb2e4103700 2024-03-23T23:47:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.117+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e402d240 con 0x7fb2e4103700 2024-03-23T23:47:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.119+0000 7fb2e0c00640 1 -- 172.21.15.99:0/2074208897 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22759 (secure 0 0 0) 0x7fb2d40bd7b0 con 0x7fb2e4103700 2024-03-23T23:47:28.119 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-23T23:47:28.120 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"5fd03d00-e96d-11ee-95cd-87774f69a715","created":"2024-03-23T23:31:55.387369+0000","modified":"2024-03-23T23:42:07.872220+0000","last_up_change":"2024-03-23T23:40:11.692029+0000","last_in_change":"2024-03-23T23:39:51.896182+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T23:35:55.293260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T23:41:08.102728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T23:41:09.219534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"819aae5f-09d9-444c-953a-ec0173dc128e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6803","nonce":1797931451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6805","nonce":1797931451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6809","nonce":1797931451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1797931451},{"type":"v1","addr":"172.21.15.99:6807","nonce":1797931451}]},"public_addr":"172.21.15.99:6803/1797931451","cluster_addr":"172.21.15.99:6805/1797931451","heartbeat_back_addr":"172.21.15.99:6809/1797931451","heartbeat_front_addr":"172.21.15.99:6807/1797931451","state":["exists","up"]},{"osd":1,"uuid":"1af37efa-acb5-4b3c-bfbd-a43df66c3893","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6801","nonce":809430535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6803","nonce":809430535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6807","nonce":809430535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":809430535},{"type":"v1","addr":"172.21.15.149:6805","nonce":809430535}]},"public_addr":"172.21.15.149:6801/809430535","cluster_addr":"172.21.15.149:6803/809430535","heartbeat_back_addr":"172.21.15.149:6807/809430535","heartbeat_front_addr":"172.21.15.149:6805/809430535","state":["exists","up"]},{"osd":2,"uuid":"6afadce7-6cd9-439a-8916-36eb8eb44c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6801","nonce":3172830280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6803","nonce":3172830280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6807","nonce":3172830280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3172830280},{"type":"v1","addr":"172.21.15.174:6805","nonce":3172830280}]},"public_addr":"172.21.15.174:6801/3172830280","cluster_addr":"172.21.15.174:6803/3172830280","heartbeat_back_addr":"172.21.15.174:6807/3172830280","heartbeat_front_addr":"172.21.15.174:6805/3172830280","state":["exists","up"]},{"osd":3,"uuid":"acb0cde2-7e55-4edf-baa5-3f6c46a83299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6811","nonce":1963358194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6813","nonce":1963358194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6817","nonce":1963358194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1963358194},{"type":"v1","addr":"172.21.15.99:6815","nonce":1963358194}]},"public_addr":"172.21.15.99:6811/1963358194","cluster_addr":"172.21.15.99:6813/1963358194","heartbeat_back_addr":"172.21.15.99:6817/1963358194","heartbeat_front_addr":"172.21.15.99:6815/1963358194","state":["exists","up"]},{"osd":4,"uuid":"2d32abb2-0d09-4f68-a5a0-2a823ec6a012","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6809","nonce":766703080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6811","nonce":766703080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6815","nonce":766703080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":766703080},{"type":"v1","addr":"172.21.15.149:6813","nonce":766703080}]},"public_addr":"172.21.15.149:6809/766703080","cluster_addr":"172.21.15.149:6811/766703080","heartbeat_back_addr":"172.21.15.149:6815/766703080","heartbeat_front_addr":"172.21.15.149:6813/766703080","state":["exists","up"]},{"osd":5,"uuid":"8f4ee3c4-a445-4abf-bf98-25a56828b679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6809","nonce":1509311749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6811","nonce":1509311749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6815","nonce":1509311749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1509311749},{"type":"v1","addr":"172.21.15.174:6813","nonce":1509311749}]},"public_addr":"172.21.15.174:6809/1509311749","cluster_addr":"172.21.15.174:6811/1509311749","heartbeat_back_addr":"172.21.15.174:6815/1509311749","heartbeat_front_addr":"172.21.15.174:6813/1509311749","state":["exists","up"]},{"osd":6,"uuid":"0a94e089-ed73-4d72-b458-74bf3589a1fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6819","nonce":1423285526}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6821","nonce":1423285526}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6825","nonce":1423285526}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1423285526},{"type":"v1","addr":"172.21.15.99:6823","nonce":1423285526}]},"public_addr":"172.21.15.99:6819/1423285526","cluster_addr":"172.21.15.99:6821/1423285526","heartbeat_back_addr":"172.21.15.99:6825/1423285526","heartbeat_front_addr":"172.21.15.99:6823/1423285526","state":["exists","up"]},{"osd":7,"uuid":"db44c8d6-8d15-4514-a5e3-0babe7b5b17b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6817","nonce":1916985722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6819","nonce":1916985722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6823","nonce":1916985722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1916985722},{"type":"v1","addr":"172.21.15.149:6821","nonce":1916985722}]},"public_addr":"172.21.15.149:6817/1916985722","cluster_addr":"172.21.15.149:6819/1916985722","heartbeat_back_addr":"172.21.15.149:6823/1916985722","heartbeat_front_addr":"172.21.15.149:6821/1916985722","state":["exists","up"]},{"osd":8,"uuid":"0f6fc144-9e8b-4ffd-91e2-7991ccb39d4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6817","nonce":2258533279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6819","nonce":2258533279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6823","nonce":2258533279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2258533279},{"type":"v1","addr":"172.21.15.174:6821","nonce":2258533279}]},"public_addr":"172.21.15.174:6817/2258533279","cluster_addr":"172.21.15.174:6819/2258533279","heartbeat_back_addr":"172.21.15.174:6823/2258533279","heartbeat_front_addr":"172.21.15.174:6821/2258533279","state":["exists","up"]},{"osd":9,"uuid":"3720287f-1023-43eb-bce3-280e52715f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6827","nonce":4103401767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6829","nonce":4103401767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6833","nonce":4103401767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":4103401767},{"type":"v1","addr":"172.21.15.99:6831","nonce":4103401767}]},"public_addr":"172.21.15.99:6827/4103401767","cluster_addr":"172.21.15.99:6829/4103401767","heartbeat_back_addr":"172.21.15.99:6833/4103401767","heartbeat_front_addr":"172.21.15.99:6831/4103401767","state":["exists","up"]},{"osd":10,"uuid":"b134a1cf-77b1-4218-b117-38efbd289712","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6825","nonce":2851714009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6827","nonce":2851714009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6831","nonce":2851714009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2851714009},{"type":"v1","addr":"172.21.15.149:6829","nonce":2851714009}]},"public_addr":"172.21.15.149:6825/2851714009","cluster_addr":"172.21.15.149:6827/2851714009","heartbeat_back_addr":"172.21.15.149:6831/2851714009","heartbeat_front_addr":"172.21.15.149:6829/2851714009","state":["exists","up"]},{"osd":11,"uuid":"b1fa75b5-9238-4a2e-bce2-4ab94569fbfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6825","nonce":2854369020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6827","nonce":2854369020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6831","nonce":2854369020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2854369020},{"type":"v1","addr":"172.21.15.174:6829","nonce":2854369020}]},"public_addr":"172.21.15.174:6825/2854369020","cluster_addr":"172.21.15.174:6827/2854369020","heartbeat_back_addr":"172.21.15.174:6831/2854369020","heartbeat_front_addr":"172.21.15.174:6829/2854369020","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:02.790548+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:27.895123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:35:51.520538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:24.046306+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:36:50.834485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:15.455918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:37:49.695319+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:17.223184+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:38:42.240785+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:16.471801+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:39:44.288628+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T23:40:10.021789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3702331558":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/1079793222":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2100135742":"2024-03-24T23:32:29.874809+0000","172.21.15.99:6800/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2517072551":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6800/2749486918":"2024-03-24T23:33:06.585425+0000","172.21.15.99:0/686293385":"2024-03-24T23:33:06.585425+0000","172.21.15.99:6801/3938208331":"2024-03-24T23:32:29.874809+0000","172.21.15.99:0/2402511247":"2024-03-24T23:32:29.874809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 msgr2=0x7fb2c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 0x7fb2c80a4cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb2e419af50 tx=0x7fb2d0009260 comp rx=0 tx=0).stop 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 msgr2=0x7fb2e4199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40349e0 tx=0x7fb2d4032430 comp rx=0 tx=0).stop 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 shutdown_connections 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fb2c80a2800 0x7fb2c80a4cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2e41052b0 0x7fb2e41a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e41048f0 0x7fb2e4199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.121+0000 7fb2e9800640 1 --2- 172.21.15.99:0/2074208897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e4103700 0x7fb2e4199a40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.122+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 >> 172.21.15.99:0/2074208897 conn(0x7fb2e40fec70 msgr2=0x7fb2e4107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.122+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 shutdown_connections 2024-03-23T23:47:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.122+0000 7fb2e9800640 1 -- 172.21.15.99:0/2074208897 wait complete. 2024-03-23T23:47:28.139 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T23:47:28.205 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-mon[31715]: pgmap v507: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:47:28.206 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2050591464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.206 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1704417382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1536248130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b81038e0 msgr2=0x7fd3b8103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1536248130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b81038e0 0x7fd3b8103d00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009900 tx=0x7fd3ac02bc30 comp rx=0 tx=0).stop 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1536248130 shutdown_connections 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1536248130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b8105490 0x7fd3b810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1536248130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 0x7fd3b8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1536248130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b81038e0 0x7fd3b8103d00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1536248130 >> 172.21.15.99:0/1536248130 conn(0x7fd3b80fee50 msgr2=0x7fd3b8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1536248130 shutdown_connections 2024-03-23T23:47:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.326+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1536248130 wait complete. 2024-03-23T23:47:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.327+0000 7fd3bfe00640 1 Processor -- start 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.327+0000 7fd3bfe00640 1 -- start start 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.327+0000 7fd3bfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.327+0000 7fd3bfe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 0x7fd3b819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8105490 0x7fd3b81a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bfe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b810f580 con 0x7fd3b8104ad0 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b810f400 con 0x7fd3b8105490 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bfe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b810f700 con 0x7fd3b81038e0 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.99:50790/0 (socket says 172.21.15.99:50790) 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 -- 172.21.15.99:0/1993957364 learned_addr learned my addr 172.21.15.99:0/1993957364 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bca00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 0x7fd3b819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bde00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8105490 0x7fd3b81a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 -- 172.21.15.99:0/1993957364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8105490 msgr2=0x7fd3b81a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8105490 0x7fd3b81a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 -- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 msgr2=0x7fd3b819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 0x7fd3b819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.328+0000 7fd3bd400640 1 -- 172.21.15.99:0/1993957364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b81a1880 con 0x7fd3b81038e0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3bd400640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000c00 tx=0x7fd3ac02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac043020 con 0x7fd3b81038e0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b81a1b10 con 0x7fd3b81038e0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac0054e0 con 0x7fd3b81038e0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003ce0 con 0x7fd3b81038e0 2024-03-23T23:47:28.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.329+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81a2080 con 0x7fd3b81038e0 2024-03-23T23:47:28.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.330+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b80688e0 con 0x7fd3b81038e0 2024-03-23T23:47:28.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.333+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd3ac046020 con 0x7fd3b81038e0 2024-03-23T23:47:28.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.335+0000 7fd3b6000640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 0x7fd3a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.335+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd3ac0f0bc0 con 0x7fd3b81038e0 2024-03-23T23:47:28.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.335+0000 7fd3bca00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 0x7fd3a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.336+0000 7fd3bca00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 0x7fd3a00a4d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400bbe0 tx=0x7fd3a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:28.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.339+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd3ac04d020 con 0x7fd3b81038e0 2024-03-23T23:47:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mon[36083]: pgmap v507: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:47:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2050591464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1704417382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mon[37002]: pgmap v507: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-23T23:47:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2050591464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1704417382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T23:47:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.598+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd3b80622a0 con 0x7fd3b81038e0 2024-03-23T23:47:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.608+0000 7fd3b6000640 1 -- 172.21.15.99:0/1993957364 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd3ac0bdbe0 con 0x7fd3b81038e0 2024-03-23T23:47:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 msgr2=0x7fd3a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 0x7fd3a00a4d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400bbe0 tx=0x7fd3a400b390 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 msgr2=0x7fd3b8199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000c00 tx=0x7fd3ac02c6c0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 shutdown_connections 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fd3a00a2880 0x7fd3a00a4d40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8105490 0x7fd3b81a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3b8104ad0 0x7fd3b819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 --2- 172.21.15.99:0/1993957364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b81038e0 0x7fd3b8199c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.611+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 >> 172.21.15.99:0/1993957364 conn(0x7fd3b80fee50 msgr2=0x7fd3b8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.612+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 shutdown_connections 2024-03-23T23:47:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.612+0000 7fd3bfe00640 1 -- 172.21.15.99:0/1993957364 wait complete. 2024-03-23T23:47:28.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1399127604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 msgr2=0x7fe2bc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1399127604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc1043f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009ca0 tx=0x7fe2b402c0f0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1399127604 shutdown_connections 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1399127604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1399127604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 0x7fe2bc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1399127604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc1043f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.819+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1399127604 >> 172.21.15.99:0/1399127604 conn(0x7fe2bc0fba10 msgr2=0x7fe2bc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:28.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.820+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1399127604 shutdown_connections 2024-03-23T23:47:28.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.820+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1399127604 wait complete. 2024-03-23T23:47:28.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.820+0000 7fe2bbe00640 1 Processor -- start 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 -- start start 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 0x7fe2bc197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc10d210 con 0x7fe2bc102000 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc10d090 con 0x7fe2bc107260 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bbe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc10d390 con 0x7fe2bc104930 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2baa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2baa00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54712/0 (socket says 172.21.15.99:54712) 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2baa00640 1 -- 172.21.15.99:0/1076263438 learned_addr learned my addr 172.21.15.99:0/1076263438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2bb400640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.821+0000 7fe2ba000640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 0x7fe2bc197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 -- 172.21.15.99:0/1076263438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 msgr2=0x7fe2bc197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 0x7fe2bc197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 -- 172.21.15.99:0/1076263438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 msgr2=0x7fe2bc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 -- 172.21.15.99:0/1076263438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc19f490 con 0x7fe2bc102000 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2bb400640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2baa00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40325f0 tx=0x7fe2b4032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4041020 con 0x7fe2bc102000 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4034cd0 con 0x7fe2bc102000 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc19f720 con 0x7fe2bc102000 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4039040 con 0x7fe2bc102000 2024-03-23T23:47:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.822+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc19fc30 con 0x7fe2bc102000 2024-03-23T23:47:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.823+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc0fd500 con 0x7fe2bc102000 2024-03-23T23:47:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.825+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe2b4044020 con 0x7fe2bc102000 2024-03-23T23:47:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.827+0000 7fe2b3400640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 0x7fe2a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.827+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8699+0+0 (secure 0 0 0) 0x7fe2b40f0a30 con 0x7fe2bc102000 2024-03-23T23:47:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.828+0000 7fe2ba000640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 0x7fe2a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.828+0000 7fe2ba000640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 0x7fe2a00a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80095f0 tx=0x7fe2a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:28.832+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2b404b020 con 0x7fe2bc102000 2024-03-23T23:47:28.861 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.861 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.861 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-mds[71201]: -- 172.21.15.99:0/4032563320 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559dec941a40 con 0x559dec914800 2024-03-23T23:47:28.861 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: starting mds.a at 2024-03-23T23:47:28.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a[31711]: 2024-03-23T23:47:28.600+0000 7f1915e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T23:47:28.862 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.862 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.862 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-mds[70842]: -- 172.21.15.99:0/3139509964 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4022a1a40 con 0x55b402275000 2024-03-23T23:47:28.862 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:47:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: starting mds.g at 2024-03-23T23:47:28.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mds[67814]: -- 172.21.15.174:0/1306701007 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aab6855a40 con 0x55aab6829000 2024-03-23T23:47:28.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: starting mds.c at 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mds[68127]: -- 172.21.15.174:0/4159051337 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a17d90ba40 con 0x55a17d8df000 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: starting mds.i at 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mds[67510]: -- 172.21.15.174:0/287328140 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f79257a40 con 0x561f7922ac00 2024-03-23T23:47:28.924 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: starting mds.f at 2024-03-23T23:47:28.925 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:28.925 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:28.925 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-mds[67195]: -- 172.21.15.174:0/382106859 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603721b1a40 con 0x560372185000 2024-03-23T23:47:28.925 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:47:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: starting mds.l at 2024-03-23T23:47:29.044 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:29.044 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:29.044 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mds[65255]: -- 172.21.15.149:0/726650757 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ca0a65a40 con 0x561ca0a38800 2024-03-23T23:47:29.044 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: starting mds.k at 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mds[65883]: -- 172.21.15.149:0/3058409893 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56283ffb5a40 con 0x56283ff88c00 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: starting mds.h at 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mds[65573]: -- 172.21.15.149:0/3041000633 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a7ceb9a40 con 0x555a7ce8c800 2024-03-23T23:47:29.045 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: starting mds.e at 2024-03-23T23:47:29.046 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: ignoring --setuser ceph since I am not root 2024-03-23T23:47:29.046 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: ignoring --setgroup ceph since I am not root 2024-03-23T23:47:29.046 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-mds[66189]: -- 172.21.15.149:0/2327535251 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f13e77a40 con 0x561f13e4ac00 2024-03-23T23:47:29.046 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:47:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: starting mds.b at 2024-03-23T23:47:29.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.064+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2bc0622a0 con 0x7fe2bc102000 2024-03-23T23:47:29.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2074208897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1993957364' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T23:47:29.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:29 smithi099 ceph-mon[31715]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2074208897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1993957364' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T23:47:29.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T23:47:29.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T23:47:29.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:29 smithi149 ceph-mon[36083]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2074208897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1993957364' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T23:47:29.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:29 smithi174 ceph-mon[37002]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-23T23:47:29.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.613+0000 7fe2b3400640 1 -- 172.21.15.99:0/1076263438 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe2b40bd930 con 0x7fe2bc102000 2024-03-23T23:47:29.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 msgr2=0x7fe2a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 0x7fe2a00a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80095f0 tx=0x7fe2a8005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 msgr2=0x7fe2bc197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40325f0 tx=0x7fe2b4032ec0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 shutdown_connections 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fe2a00a2880 0x7fe2a00a4d40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2bc107260 0x7fe2bc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2bc104930 0x7fe2bc197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 --2- 172.21.15.99:0/1076263438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2bc102000 0x7fe2bc197870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 >> 172.21.15.99:0/1076263438 conn(0x7fe2bc0fba10 msgr2=0x7fe2bc102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.617+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 shutdown_connections 2024-03-23T23:47:29.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.618+0000 7fe2bbe00640 1 -- 172.21.15.99:0/1076263438 wait complete. 2024-03-23T23:47:29.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T23:47:29.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3188839576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0076060 msgr2=0x7fbcc0076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:29.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3188839576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0076060 0x7fbcc0076480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009900 tx=0x7fbcb402bc30 comp rx=0 tx=0).stop 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3188839576 shutdown_connections 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3188839576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc0077c40 0x7fbcc00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3188839576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc00772a0 0x7fbcc0077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3188839576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0076060 0x7fbcc0076480 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3188839576 >> 172.21.15.99:0/3188839576 conn(0x7fbcc00fee50 msgr2=0x7fbcc0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3188839576 shutdown_connections 2024-03-23T23:47:29.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.834+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3188839576 wait complete. 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 Processor -- start 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 -- start start 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 0x7fbcc019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01139b0 con 0x7fbcc0077c40 2024-03-23T23:47:29.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0113830 con 0x7fbcc00772a0 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.835+0000 7fbcc7a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0113b30 con 0x7fbcc0076060 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 0x7fbcc019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.99:55172/0 (socket says 172.21.15.99:55172) 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60986/0 (socket says 172.21.15.99:60986) 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcbfe00640 1 -- 172.21.15.99:0/3145214162 learned_addr learned my addr 172.21.15.99:0/3145214162 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 -- 172.21.15.99:0/3145214162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 msgr2=0x7fbcc019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 -- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 msgr2=0x7fbcc01a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 -- 172.21.15.99:0/3145214162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc01a5c20 con 0x7fbcc0076060 2024-03-23T23:47:29.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcbfe00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.836+0000 7fbcc5000640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 0x7fbcc019e0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4036040 tx=0x7fbcb402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.837+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4043020 con 0x7fbcc0076060 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.837+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbcb40054e0 con 0x7fbcc0076060 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.837+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc01a5eb0 con 0x7fbcc0076060 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.837+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4003c30 con 0x7fbcc0076060 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.837+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc01a6310 con 0x7fbcc0076060 2024-03-23T23:47:29.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.838+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc0110490 con 0x7fbcc0076060 2024-03-23T23:47:29.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.840+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbcb4046020 con 0x7fbcc0076060 2024-03-23T23:47:29.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.841+0000 7fbcbd600640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 0x7fbca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:29.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.841+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8699+0+0 (secure 0 0 0) 0x7fbcb40f12a0 con 0x7fbcc0076060 2024-03-23T23:47:29.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.841+0000 7fbcbfe00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 0x7fbca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:29.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.842+0000 7fbcbfe00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 0x7fbca80a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbcc019f510 tx=0x7fbcac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:29.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:29.843+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbcb404d020 con 0x7fbcc0076060 2024-03-23T23:47:30.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.042+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbcc002d070 con 0x7fbcc0076060 2024-03-23T23:47:30.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: pgmap v509: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: Cluster is now healthy 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6832/3449634992,v1:172.21.15.174:6834/3449634992] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6836/4294863610,v1:172.21.15.174:6838/4294863610] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6834/4167179125,v1:172.21.15.149:6835/4167179125] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6833/3332941460,v1:172.21.15.174:6835/3332941460] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6832/3874120068,v1:172.21.15.149:6833/3874120068] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6837/677541502,v1:172.21.15.149:6839/677541502] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.149:6836/91652694,v1:172.21.15.149:6838/91652694] up:boot 2024-03-23T23:47:30.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.174:6837/2642116708,v1:172.21.15.174:6839/2642116708] up:boot 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6838/3373865655,v1:172.21.15.99:6840/3373865655] up:boot 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: mds.? [v2:172.21.15.99:6839/2939506401,v1:172.21.15.99:6841/2939506401] up:boot 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: fsmap 12 up:standby 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:47:30.365 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:30 smithi099 ceph-mon[31715]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:47:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: pgmap v509: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-23T23:47:30.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: Cluster is now healthy 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6832/3449634992,v1:172.21.15.174:6834/3449634992] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6836/4294863610,v1:172.21.15.174:6838/4294863610] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6834/4167179125,v1:172.21.15.149:6835/4167179125] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6833/3332941460,v1:172.21.15.174:6835/3332941460] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6832/3874120068,v1:172.21.15.149:6833/3874120068] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6837/677541502,v1:172.21.15.149:6839/677541502] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.149:6836/91652694,v1:172.21.15.149:6838/91652694] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.174:6837/2642116708,v1:172.21.15.174:6839/2642116708] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6838/3373865655,v1:172.21.15.99:6840/3373865655] up:boot 2024-03-23T23:47:30.395 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: mds.? [v2:172.21.15.99:6839/2939506401,v1:172.21.15.99:6841/2939506401] up:boot 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: fsmap 12 up:standby 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:47:30.396 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:30 smithi149 ceph-mon[36083]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: pgmap v509: 97 pgs: 97 active+clean; 8.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: Cluster is now healthy 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/1076263438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T23:47:30.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6832/3449634992,v1:172.21.15.174:6834/3449634992] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6836/4294863610,v1:172.21.15.174:6838/4294863610] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6834/4167179125,v1:172.21.15.149:6835/4167179125] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6833/3332941460,v1:172.21.15.174:6835/3332941460] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6832/3874120068,v1:172.21.15.149:6833/3874120068] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6837/677541502,v1:172.21.15.149:6839/677541502] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.149:6836/91652694,v1:172.21.15.149:6838/91652694] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.174:6837/2642116708,v1:172.21.15.174:6839/2642116708] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6838/3373865655,v1:172.21.15.99:6840/3373865655] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: mds.? [v2:172.21.15.99:6839/2939506401,v1:172.21.15.99:6841/2939506401] up:boot 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: fsmap 12 up:standby 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T23:47:30.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T23:47:30.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T23:47:30.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T23:47:30.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T23:47:30.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T23:47:30.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:30 smithi174 ceph-mon[37002]: from='mgr.14152 172.21.15.99:0/2292663078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T23:47:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.621+0000 7fbcbd600640 1 -- 172.21.15.99:0/3145214162 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbcb40be110 con 0x7fbcc0076060 2024-03-23T23:47:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 msgr2=0x7fbca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 0x7fbca80a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbcc019f510 tx=0x7fbcac005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 msgr2=0x7fbcc019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 0x7fbcc019e0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4036040 tx=0x7fbcb402c650 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 shutdown_connections 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7fbca80a2800 0x7fbca80a4cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcc0077c40 0x7fbcc01a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00772a0 0x7fbcc019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 --2- 172.21.15.99:0/3145214162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcc0076060 0x7fbcc019e0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.629+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 >> 172.21.15.99:0/3145214162 conn(0x7fbcc00fee50 msgr2=0x7fbcc0100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.630+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 shutdown_connections 2024-03-23T23:47:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.630+0000 7fbcc7a00640 1 -- 172.21.15.99:0/3145214162 wait complete. 2024-03-23T23:47:30.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 -- 172.21.15.99:0/927667938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c1038e0 msgr2=0x7f334c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 --2- 172.21.15.99:0/927667938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c1038e0 0x7f334c103d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f333c009900 tx=0x7f333c02bc40 comp rx=0 tx=0).stop 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 -- 172.21.15.99:0/927667938 shutdown_connections 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 --2- 172.21.15.99:0/927667938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c105490 0x7f334c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 --2- 172.21.15.99:0/927667938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c104ad0 0x7f334c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 --2- 172.21.15.99:0/927667938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c1038e0 0x7f334c103d00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.864+0000 7f3353200640 1 -- 172.21.15.99:0/927667938 >> 172.21.15.99:0/927667938 conn(0x7f334c0fee50 msgr2=0x7f334c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.865+0000 7f3353200640 1 -- 172.21.15.99:0/927667938 shutdown_connections 2024-03-23T23:47:30.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.865+0000 7f3353200640 1 -- 172.21.15.99:0/927667938 wait complete. 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.865+0000 7f3353200640 1 Processor -- start 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 -- start start 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 0x7f334c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c105490 0x7f334c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:30.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c10f5b0 con 0x7f334c1038e0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c10f430 con 0x7f334c105490 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3353200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c10f730 con 0x7f334c104ad0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f334be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f334be00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32776/0 (socket says 172.21.15.99:32776) 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f334be00640 1 -- 172.21.15.99:0/2655215580 learned_addr learned my addr 172.21.15.99:0/2655215580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3343400640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 0x7f334c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.866+0000 7f3351200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c105490 0x7f334c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 -- 172.21.15.99:0/2655215580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 msgr2=0x7f334c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 0x7f334c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 -- 172.21.15.99:0/2655215580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c105490 msgr2=0x7f334c1a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c105490 0x7f334c1a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 -- 172.21.15.99:0/2655215580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c1a1850 con 0x7f334c1038e0 2024-03-23T23:47:30.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f3343400640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 0x7f334c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T23:47:30.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f334be00640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f333c02efd0 tx=0x7f333c02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:30.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.867+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c041070 con 0x7f334c1038e0 2024-03-23T23:47:30.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.868+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f333c004450 con 0x7f334c1038e0 2024-03-23T23:47:30.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.868+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c1a1ae0 con 0x7f334c1038e0 2024-03-23T23:47:30.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.868+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c003e50 con 0x7f334c1038e0 2024-03-23T23:47:30.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.868+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c1a1fa0 con 0x7f334c1038e0 2024-03-23T23:47:30.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.869+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c10ac50 con 0x7f334c1038e0 2024-03-23T23:47:30.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.871+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f333c044070 con 0x7f334c1038e0 2024-03-23T23:47:30.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.872+0000 7f3349600640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 0x7f33300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T23:47:30.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.872+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8145+0+0 (secure 0 0 0) 0x7f333c0f05c0 con 0x7f334c1038e0 2024-03-23T23:47:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.873+0000 7f3343400640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 0x7f33300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T23:47:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.873+0000 7f3343400640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 0x7f33300a4e60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f33340095f0 tx=0x7f3334005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T23:47:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:30.877+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f333c0bd0b0 con 0x7f334c1038e0 2024-03-23T23:47:31.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:31.077+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f334c02d070 con 0x7f334c1038e0 2024-03-23T23:47:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:31 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/3145214162' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:31 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:31 smithi099 ceph-mon[31715]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T23:47:31.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:31 smithi099 ceph-mon[31715]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T23:47:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:31 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/3145214162' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:31 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:31 smithi149 ceph-mon[36083]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T23:47:31.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:31 smithi149 ceph-mon[36083]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T23:47:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:31 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/3145214162' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:31 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:31 smithi174 ceph-mon[37002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T23:47:31.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:31 smithi174 ceph-mon[37002]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T23:47:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.061+0000 7f3349600640 1 -- 172.21.15.99:0/2655215580 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f333c0c1150 con 0x7f334c1038e0 2024-03-23T23:47:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-03-23T23:47:32.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.065+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 msgr2=0x7f33300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:32.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.065+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 0x7f33300a4e60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f33340095f0 tx=0x7f3334005f90 comp rx=0 tx=0).stop 2024-03-23T23:47:32.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.065+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 msgr2=0x7f334c199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T23:47:32.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.065+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f333c02efd0 tx=0x7f333c02c660 comp rx=0 tx=0).stop 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 shutdown_connections 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:6800/3957482815,v1:172.21.15.99:6801/3957482815] conn(0x7f33300a29a0 0x7f33300a4e60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f334c105490 0x7f334c1a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f334c104ad0 0x7f334c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 --2- 172.21.15.99:0/2655215580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334c1038e0 0x7f334c199c40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T23:47:32.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.066+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 >> 172.21.15.99:0/2655215580 conn(0x7f334c0fee50 msgr2=0x7f334c107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T23:47:32.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.068+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 shutdown_connections 2024-03-23T23:47:32.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-23T23:47:32.068+0000 7f3353200640 1 -- 172.21.15.99:0/2655215580 wait complete. 2024-03-23T23:47:32.095 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:47:32.166 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:47:32.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:47:32.237 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-23T23:47:32.237 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T23:47:32.293 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T23:47:32.310 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T23:47:32.328 INFO:tasks.cephadm:Stopping all daemons... 2024-03-23T23:47:32.328 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-23T23:47:32.328 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a 2024-03-23T23:47:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:32 smithi099 ceph-mon[31715]: pgmap v511: 65 pgs: 65 active+clean; 577 KiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:47:32.361 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:32 smithi099 ceph-mon[31715]: from='client.? 172.21.15.99:0/2655215580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:32 smithi149 ceph-mon[36083]: pgmap v511: 65 pgs: 65 active+clean; 577 KiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:47:32.394 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:32 smithi149 ceph-mon[36083]: from='client.? 172.21.15.99:0/2655215580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:32.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:32 smithi174 ceph-mon[37002]: pgmap v511: 65 pgs: 65 active+clean; 577 KiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T23:47:32.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:32 smithi174 ceph-mon[37002]: from='client.? 172.21.15.99:0/2655215580' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T23:47:32.862 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:32 smithi099 systemd[1]: Stopping Ceph mon.a for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:33.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a[31711]: 2024-03-23T23:47:32.904+0000 7f191cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:33.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a[31711]: 2024-03-23T23:47:32.904+0000 7f191cc00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-23T23:47:33.362 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:33 smithi099 podman[88313]: 2024-03-23 23:47:33.153356043 +0000 UTC m=+0.391859875 container died 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-23T23:47:34.111 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:33 smithi099 podman[88313]: 2024-03-23 23:47:33.786906456 +0000 UTC m=+1.025410298 container cleanup 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-23T23:47:34.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:33 smithi099 bash[88313]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a 2024-03-23T23:47:34.112 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 23 23:47:34 smithi099 podman[88326]: 2024-03-23 23:47:34.034738292 +0000 UTC m=+0.865988690 container remove 57a017c5170b913d08975d0ffda18485a1b1842a131b0d7ae41e07a56e2bc9f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:47:34.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.a.service' 2024-03-23T23:47:34.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:34.682 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-23T23:47:34.682 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-23T23:47:34.682 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.b 2024-03-23T23:47:34.993 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:34 smithi149 systemd[1]: Stopping Ceph mon.b for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:35.310 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-b[36079]: 2024-03-23T23:47:34.992+0000 7f5bf9f6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:35.310 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-b[36079]: 2024-03-23T23:47:34.992+0000 7f5bf9f6c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-23T23:47:35.310 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:35 smithi149 podman[69272]: 2024-03-23 23:47:35.023847788 +0000 UTC m=+0.146651962 container died 1f9bb8b463998ad128550ab48c1b34ba5abb4fca32d9ac254a5647e61ef2b7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-23T23:47:35.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:35 smithi149 podman[69272]: 2024-03-23 23:47:35.402233943 +0000 UTC m=+0.525038140 container cleanup 1f9bb8b463998ad128550ab48c1b34ba5abb4fca32d9ac254a5647e61ef2b7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T23:47:35.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 23 23:47:35 smithi149 bash[69272]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-b 2024-03-23T23:47:35.858 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.b.service' 2024-03-23T23:47:35.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:35.909 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-23T23:47:35.909 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-23T23:47:35.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.c 2024-03-23T23:47:36.230 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:35 smithi174 systemd[1]: Stopping Ceph mon.c for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:36.657 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-c[36998]: 2024-03-23T23:47:36.230+0000 7f0d9b821700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:36.658 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-c[36998]: 2024-03-23T23:47:36.230+0000 7f0d9b821700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-23T23:47:36.658 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:36 smithi174 podman[71154]: 2024-03-23 23:47:36.363254749 +0000 UTC m=+0.255053613 container died 863c6d10876aca4fbd14f8738af8120e98ad2f9b8b83dd3438ca2805e03559f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:47:36.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:36 smithi174 podman[71154]: 2024-03-23 23:47:36.764547703 +0000 UTC m=+0.656346575 container cleanup 863c6d10876aca4fbd14f8738af8120e98ad2f9b8b83dd3438ca2805e03559f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-23T23:47:36.923 INFO:journalctl@ceph.mon.c.smithi174.stdout:Mar 23 23:47:36 smithi174 bash[71154]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mon-c 2024-03-23T23:47:37.190 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mon.c.service' 2024-03-23T23:47:37.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:37.263 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-23T23:47:37.264 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-23T23:47:37.264 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x 2024-03-23T23:47:37.611 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:37 smithi099 systemd[1]: Stopping Ceph mgr.x for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:38.361 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:38 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x[31946]: 2024-03-23T23:47:38.030+0000 7f705f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:38.362 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:38 smithi099 podman[88434]: 2024-03-23 23:47:38.124597584 +0000 UTC m=+0.436077025 container died 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-23T23:47:39.111 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:38 smithi099 podman[88434]: 2024-03-23 23:47:38.738757271 +0000 UTC m=+1.050236751 container cleanup 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-23T23:47:39.112 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:38 smithi099 bash[88434]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x 2024-03-23T23:47:39.112 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 23 23:47:38 smithi099 podman[88449]: 2024-03-23 23:47:38.974250624 +0000 UTC m=+0.837388139 container remove 71f4dee37fba3c196076c3cbe2a716d0bbef68aa558481146192ac35504ef641 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:47:39.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-23T23:47:39.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:39.636 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-23T23:47:39.636 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-23T23:47:39.636 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.y 2024-03-23T23:47:39.969 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:39 smithi149 systemd[1]: Stopping Ceph mgr.y for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:40.232 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:39 smithi149 podman[69395]: 2024-03-23 23:47:39.969190268 +0000 UTC m=+0.141561327 container died 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319) 2024-03-23T23:47:40.617 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:40 smithi149 podman[69395]: 2024-03-23 23:47:40.390638577 +0000 UTC m=+0.563009629 container cleanup 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-23T23:47:40.617 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:40 smithi149 bash[69395]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y 2024-03-23T23:47:40.868 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:40 smithi149 podman[69410]: 2024-03-23 23:47:40.617482485 +0000 UTC m=+0.644602413 container remove 3b22133feda86f9b0fac1f22eed0e3df353a411927aa4dcb3dcf242c6a4401d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-23T23:47:40.868 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 23 23:47:40 smithi149 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-23T23:47:40.872 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-23T23:47:40.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:40.912 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-23T23:47:40.912 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-23T23:47:40.912 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.z 2024-03-23T23:47:41.248 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:47:40 smithi174 systemd[1]: Stopping Ceph mgr.z for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:41.544 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:47:41 smithi174 podman[71276]: 2024-03-23 23:47:41.248456431 +0000 UTC m=+0.143320567 container died a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:47:41.887 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:47:41 smithi174 podman[71276]: 2024-03-23 23:47:41.644409736 +0000 UTC m=+0.539273856 container cleanup a1ba38b568eb0986f0db0f24727526dddfa17c9276cea914ac9e6654207381b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T23:47:41.887 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Mar 23 23:47:41 smithi174 bash[71276]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mgr-z 2024-03-23T23:47:42.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-23T23:47:42.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:42.099 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-23T23:47:42.100 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-23T23:47:42.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.0 2024-03-23T23:47:42.611 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:42 smithi099 systemd[1]: Stopping Ceph osd.0 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:43.111 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0[44273]: 2024-03-23T23:47:42.700+0000 7f75b4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:43.112 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0[44273]: 2024-03-23T23:47:42.701+0000 7f75b4c00700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-23T23:47:43.112 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0[44273]: 2024-03-23T23:47:42.701+0000 7f75b4c00700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:47:48.574 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:48 smithi099 podman[88555]: 2024-03-23 23:47:48.198914974 +0000 UTC m=+5.659851010 container died e0e1a505ad9f443f54325f9c9bde50299e14b06b32916ff1aa65863ed0799336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:47:48.861 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:48 smithi099 podman[88555]: 2024-03-23 23:47:48.786254608 +0000 UTC m=+6.247190673 container cleanup e0e1a505ad9f443f54325f9c9bde50299e14b06b32916ff1aa65863ed0799336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-23T23:47:48.861 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:48 smithi099 bash[88555]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0 2024-03-23T23:47:49.361 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:49 smithi099 podman[88574]: 2024-03-23 23:47:49.061296668 +0000 UTC m=+0.842644875 container remove e0e1a505ad9f443f54325f9c9bde50299e14b06b32916ff1aa65863ed0799336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-23T23:47:50.249 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:49 smithi099 podman[88663]: 2024-03-23 23:47:49.798661424 +0000 UTC m=+0.036971079 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:47:50.249 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:49 smithi099 podman[88663]: 2024-03-23 23:47:49.928864684 +0000 UTC m=+0.167174375 container create 0e0527bc48d9b0742528b551988a6e4ec1069d1cfdeb66e59b67f73e3617f750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-23T23:47:50.611 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:50 smithi099 podman[88663]: 2024-03-23 23:47:50.249679945 +0000 UTC m=+0.487989643 container init 0e0527bc48d9b0742528b551988a6e4ec1069d1cfdeb66e59b67f73e3617f750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-23T23:47:50.611 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:50 smithi099 podman[88663]: 2024-03-23 23:47:50.261140691 +0000 UTC m=+0.499450389 container start 0e0527bc48d9b0742528b551988a6e4ec1069d1cfdeb66e59b67f73e3617f750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:47:50.611 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:50 smithi099 podman[88663]: 2024-03-23 23:47:50.365372516 +0000 UTC m=+0.603682251 container attach 0e0527bc48d9b0742528b551988a6e4ec1069d1cfdeb66e59b67f73e3617f750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:47:51.061 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 23 23:47:50 smithi099 podman[88663]: 2024-03-23 23:47:50.692834217 +0000 UTC m=+0.931143935 container died 0e0527bc48d9b0742528b551988a6e4ec1069d1cfdeb66e59b67f73e3617f750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:47:51.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.0.service' 2024-03-23T23:47:51.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:51.759 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-23T23:47:51.759 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-23T23:47:51.759 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.1 2024-03-23T23:47:52.078 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:51 smithi149 systemd[1]: Stopping Ceph osd.1 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:47:52.394 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1[40667]: 2024-03-23T23:47:52.078+0000 7f3b86301700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:47:52.394 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1[40667]: 2024-03-23T23:47:52.078+0000 7f3b86301700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-23T23:47:52.394 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1[40667]: 2024-03-23T23:47:52.078+0000 7f3b86301700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:47:57.894 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:57 smithi149 podman[69535]: 2024-03-23 23:47:57.581540886 +0000 UTC m=+5.627358329 container died 5b121b74a7811896facab378510d01c84f97cc0af6c9dfa8d3ca1a460851b5ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:47:58.222 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:57 smithi149 podman[69535]: 2024-03-23 23:47:57.970347227 +0000 UTC m=+6.016164661 container cleanup 5b121b74a7811896facab378510d01c84f97cc0af6c9dfa8d3ca1a460851b5ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:47:58.222 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:57 smithi149 bash[69535]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1 2024-03-23T23:47:58.532 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69552]: 2024-03-23 23:47:58.221690195 +0000 UTC m=+0.635672362 container remove 5b121b74a7811896facab378510d01c84f97cc0af6c9dfa8d3ca1a460851b5ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:47:58.786 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69641]: 2024-03-23 23:47:58.436740247 +0000 UTC m=+0.017840942 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:47:58.786 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69641]: 2024-03-23 23:47:58.54760124 +0000 UTC m=+0.128701920 container create 594c367e55728c34efbefc28b28f7c9848fd1e27b73a3ee1b2fb1641abc93473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-23T23:47:58.786 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69641]: 2024-03-23 23:47:58.783637804 +0000 UTC m=+0.364738492 container init 594c367e55728c34efbefc28b28f7c9848fd1e27b73a3ee1b2fb1641abc93473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:47:58.786 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69641]: 2024-03-23 23:47:58.786767573 +0000 UTC m=+0.367868251 container start 594c367e55728c34efbefc28b28f7c9848fd1e27b73a3ee1b2fb1641abc93473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-23T23:47:59.074 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:58 smithi149 podman[69641]: 2024-03-23 23:47:58.883841469 +0000 UTC m=+0.464942168 container attach 594c367e55728c34efbefc28b28f7c9848fd1e27b73a3ee1b2fb1641abc93473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-23T23:47:59.345 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 23 23:47:59 smithi149 podman[69641]: 2024-03-23 23:47:59.07522775 +0000 UTC m=+0.656328435 container died 594c367e55728c34efbefc28b28f7c9848fd1e27b73a3ee1b2fb1641abc93473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-23T23:47:59.785 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.1.service' 2024-03-23T23:47:59.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:47:59.826 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-23T23:47:59.826 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-23T23:47:59.826 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.2 2024-03-23T23:48:00.139 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:47:59 smithi174 systemd[1]: Stopping Ceph osd.2 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:00.399 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:48:00.139+0000 7fdb8d0c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:00.399 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:48:00.139+0000 7fdb8d0c6700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-23T23:48:00.399 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2[42151]: 2024-03-23T23:48:00.139+0000 7fdb8d0c6700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:05.882 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:05 smithi174 podman[71401]: 2024-03-23 23:48:05.604137882 +0000 UTC m=+5.589569113 container died e510c03b241e16930375868b7bf0d28323461a93b113870197810058dee00af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:06.174 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:05 smithi174 podman[71401]: 2024-03-23 23:48:05.964881349 +0000 UTC m=+5.950312579 container cleanup e510c03b241e16930375868b7bf0d28323461a93b113870197810058dee00af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-23T23:48:06.174 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:05 smithi174 bash[71401]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2 2024-03-23T23:48:06.581 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71430]: 2024-03-23 23:48:06.183014474 +0000 UTC m=+0.577060497 container remove e510c03b241e16930375868b7bf0d28323461a93b113870197810058dee00af8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T23:48:06.861 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71521]: 2024-03-23 23:48:06.484955795 +0000 UTC m=+0.017795222 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:06.861 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71521]: 2024-03-23 23:48:06.642331911 +0000 UTC m=+0.175171344 container create b2a84bd1b3419d1e3c436463e4410d2bd40f9f7d3dd8a69c32bccc111e4ef9dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:06.861 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71521]: 2024-03-23 23:48:06.861612753 +0000 UTC m=+0.394452180 container init b2a84bd1b3419d1e3c436463e4410d2bd40f9f7d3dd8a69c32bccc111e4ef9dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:07.160 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71521]: 2024-03-23 23:48:06.864789523 +0000 UTC m=+0.397628945 container start b2a84bd1b3419d1e3c436463e4410d2bd40f9f7d3dd8a69c32bccc111e4ef9dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:07.160 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:06 smithi174 podman[71521]: 2024-03-23 23:48:06.936624411 +0000 UTC m=+0.469463851 container attach b2a84bd1b3419d1e3c436463e4410d2bd40f9f7d3dd8a69c32bccc111e4ef9dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:07.424 INFO:journalctl@ceph.osd.2.smithi174.stdout:Mar 23 23:48:07 smithi174 podman[71521]: 2024-03-23 23:48:07.161219685 +0000 UTC m=+0.694059106 container died b2a84bd1b3419d1e3c436463e4410d2bd40f9f7d3dd8a69c32bccc111e4ef9dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-23T23:48:07.809 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.2.service' 2024-03-23T23:48:07.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:07.857 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-23T23:48:07.857 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-23T23:48:07.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.3 2024-03-23T23:48:08.362 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:07 smithi099 systemd[1]: Stopping Ceph osd.3 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:08.862 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:08 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:08.469+0000 7fdd45a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:08.862 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:08 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:08.469+0000 7fdd45a00700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-23T23:48:08.862 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:08 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:08.469+0000 7fdd45a00700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:09.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:09 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:09.069+0000 7fdd3c200700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.662224+0000 front 2024-03-23T23:47:46.662533+0000 (oldest deadline 2024-03-23T23:48:08.961836+0000) 2024-03-23T23:48:09.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:09.283+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:10.058 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:09 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:09.945+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:10.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:10.056+0000 7fdd3c200700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.662224+0000 front 2024-03-23T23:47:46.662533+0000 (oldest deadline 2024-03-23T23:48:08.961836+0000) 2024-03-23T23:48:10.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:10.080+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:10.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:10.278+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:11.035 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:10 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:10.910+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:11.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:11.034+0000 7fdd3c200700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.662224+0000 front 2024-03-23T23:47:46.662533+0000 (oldest deadline 2024-03-23T23:48:08.961836+0000) 2024-03-23T23:48:11.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:11.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:11.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:11.295+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:12.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:11.899+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:12.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:11 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:11.958+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:12.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:12 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:12.063+0000 7fdd3c200700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.662224+0000 front 2024-03-23T23:47:46.662533+0000 (oldest deadline 2024-03-23T23:48:08.961836+0000) 2024-03-23T23:48:12.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:12 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:12.053+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:12.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:12.248+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:13.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:12.812+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:13.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:12.857+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:13.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:12 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:12.988+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:13.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:13 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3[51092]: 2024-03-23T23:48:13.090+0000 7fdd3c200700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.662224+0000 front 2024-03-23T23:47:46.662533+0000 (oldest deadline 2024-03-23T23:48:08.961836+0000) 2024-03-23T23:48:13.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:13 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:13.039+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:13.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:13 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:13.231+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:14.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:13.861+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:14.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:13.857+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:14.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:13 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:13.986+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:14.362 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:13 smithi099 podman[88767]: 2024-03-23 23:48:13.970844489 +0000 UTC m=+5.690812437 container died e179d3108dd47530d7216bd7b3f5066a9f882d52966c2b70c4922f2a45009d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:14.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:14 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:14.046+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:14.625 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:14 smithi099 podman[88767]: 2024-03-23 23:48:14.606156673 +0000 UTC m=+6.326124651 container cleanup e179d3108dd47530d7216bd7b3f5066a9f882d52966c2b70c4922f2a45009d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:14.626 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:14 smithi099 bash[88767]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3 2024-03-23T23:48:14.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:14 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:14.239+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:15.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:14.858+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:15.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:14.846+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:15.361 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:14 smithi099 podman[88797]: 2024-03-23 23:48:14.882852606 +0000 UTC m=+0.899103403 container remove e179d3108dd47530d7216bd7b3f5066a9f882d52966c2b70c4922f2a45009d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-23T23:48:15.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:15 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:15.003+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:15.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:15 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:15.079+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:15.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:15 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:15.245+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:16.030 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:15 smithi099 podman[88889]: 2024-03-23 23:48:15.557329304 +0000 UTC m=+0.034367774 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:16.030 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:15 smithi099 podman[88889]: 2024-03-23 23:48:15.700516086 +0000 UTC m=+0.177554587 container create 900f5b6a87cd97135b49e62f17931e6bbf7739eed9bf408586a32e17dfb49379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-23T23:48:16.030 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:16 smithi099 podman[88889]: 2024-03-23 23:48:16.029469729 +0000 UTC m=+0.506508237 container init 900f5b6a87cd97135b49e62f17931e6bbf7739eed9bf408586a32e17dfb49379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:16.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:15.887+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:16.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:15.889+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:16.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:16.090+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:16.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:16 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:16.040+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:16.362 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:16 smithi099 podman[88889]: 2024-03-23 23:48:16.042275775 +0000 UTC m=+0.519314249 container start 900f5b6a87cd97135b49e62f17931e6bbf7739eed9bf408586a32e17dfb49379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-23T23:48:16.362 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:16 smithi099 podman[88889]: 2024-03-23 23:48:16.145198521 +0000 UTC m=+0.622237028 container attach 900f5b6a87cd97135b49e62f17931e6bbf7739eed9bf408586a32e17dfb49379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T23:48:16.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:16 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:16.259+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:16.861 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 23 23:48:16 smithi099 podman[88889]: 2024-03-23 23:48:16.470942602 +0000 UTC m=+0.947981078 container died 900f5b6a87cd97135b49e62f17931e6bbf7739eed9bf408586a32e17dfb49379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-23T23:48:17.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:16.926+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:17.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:16.881+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:17.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:17 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:17.086+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:17.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:17 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:17.018+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:17.469 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.3.service' 2024-03-23T23:48:17.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:17.594 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-23T23:48:17.594 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-23T23:48:17.594 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.4 2024-03-23T23:48:17.621 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:17 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:17.273+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:17.894 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:17 smithi149 systemd[1]: Stopping Ceph osd.4 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:18.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:17.934+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:18.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:17.849+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:18.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:17.849+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:18.283 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:17 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:48:17.906+0000 7f7721668700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:18.283 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:17 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:48:17.906+0000 7f7721668700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-23T23:48:18.283 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:17 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:48:17.906+0000 7f7721668700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:18.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:18 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:18.046+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:18.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:18 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:18.127+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:18.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:18 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:18.283+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:19.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:18.894+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:19.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:18.832+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:19.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:18.832+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:19.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:19 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:19.076+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:19.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:19 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:19.134+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:19.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:19 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:19.134+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:19.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:19 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:19.278+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:20.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:19.920+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:20.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:19.849+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:20.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:19.849+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:20.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:20 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:20.075+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:20.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:20 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:20.124+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:20.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:20 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:20.124+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:20.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:20 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:20.302+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:21.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:20.941+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:21.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:20.874+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:21.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:20.874+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:21.611 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:21.124+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:21.611 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:21.124+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:21.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:21.169+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:21.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:21 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:21.169+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:21.644 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:21 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:48:21.289+0000 7f7719c54700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.546666+0000 front 2024-03-23T23:47:56.546641+0000 (oldest deadline 2024-03-23T23:48:20.646879+0000) 2024-03-23T23:48:21.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:21 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:21.289+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:22.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:21 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:21.897+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:22.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:21 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:21.926+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:22.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:21 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:21.855+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:22.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:21 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:21.856+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:22.611 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:22.122+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:22.611 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:22.122+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:22.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:22.162+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:22.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:22 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:22.162+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:22.644 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:22 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4[47202]: 2024-03-23T23:48:22.267+0000 7f7719c54700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.546666+0000 front 2024-03-23T23:47:56.546641+0000 (oldest deadline 2024-03-23T23:48:20.646879+0000) 2024-03-23T23:48:22.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:22 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:22.279+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:22.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:22 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:22.248+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:22.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:22 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:22.248+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:23.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:22 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:22.873+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:23.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:22 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:22.878+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:23.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:22 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:22.880+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:23.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:22 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:22.880+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:23.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:23.083+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:23.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:23.083+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:23.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:23.118+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:23.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:23.118+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:23.558 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:23 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:23.289+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:23.559 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:23 smithi149 podman[69758]: 2024-03-23 23:48:23.248664775 +0000 UTC m=+5.461316421 container died ff66324ef80fc4ef2715929b687c22ba9e7268c4bebe76690b5893dd647f51c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:23.559 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:23 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:23.208+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:23.560 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:23 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:23.208+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:23.874 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:23 smithi149 podman[69758]: 2024-03-23 23:48:23.656825028 +0000 UTC m=+5.869476676 container cleanup ff66324ef80fc4ef2715929b687c22ba9e7268c4bebe76690b5893dd647f51c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-23T23:48:23.874 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:23 smithi149 bash[69758]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4 2024-03-23T23:48:24.144 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:23 smithi149 podman[69775]: 2024-03-23 23:48:23.874767579 +0000 UTC m=+0.626028738 container remove ff66324ef80fc4ef2715929b687c22ba9e7268c4bebe76690b5893dd647f51c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:24.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:23 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:23.886+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:24.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:23 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:23.840+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:24.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:23 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:23.840+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:24.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:23 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:23.840+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:24.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:23 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:23.840+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:24.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:24.074+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:24.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:24.074+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:24.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:24.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:24.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:24 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:24.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:24.502 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:24 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:24.330+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:24.503 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:24 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:24.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:24.503 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:24 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:24.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:24.503 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.172963222 +0000 UTC m=+0.018005492 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:24.503 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.275650813 +0000 UTC m=+0.120693059 container create 4eb627d651ad2a157589829fb03f104c57a466810e48d049e4cd5efe957dd3de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T23:48:24.503 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.503300403 +0000 UTC m=+0.348342670 container init 4eb627d651ad2a157589829fb03f104c57a466810e48d049e4cd5efe957dd3de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T23:48:24.789 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.50664053 +0000 UTC m=+0.351682792 container start 4eb627d651ad2a157589829fb03f104c57a466810e48d049e4cd5efe957dd3de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-23T23:48:24.789 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.603416939 +0000 UTC m=+0.448459204 container attach 4eb627d651ad2a157589829fb03f104c57a466810e48d049e4cd5efe957dd3de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-23T23:48:25.144 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 23 23:48:24 smithi149 podman[69872]: 2024-03-23 23:48:24.790093172 +0000 UTC m=+0.635135430 container died 4eb627d651ad2a157589829fb03f104c57a466810e48d049e4cd5efe957dd3de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:25.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:24 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:24.843+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:25.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:24 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:24.805+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:25.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:24 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:24.805+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:25.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:24 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:24.829+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:25.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:24 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:24.829+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:25.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:25.101+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:25.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:25.101+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:25.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:25.071+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:25.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:25 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:25.071+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:25.473 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:25 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:25.299+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:25.473 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:25 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:25.202+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:25.473 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:25 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:25.202+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:25.499 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.4.service' 2024-03-23T23:48:25.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:25.583 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-23T23:48:25.584 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-23T23:48:25.584 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.5 2024-03-23T23:48:25.868 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 systemd[1]: Stopping Ceph osd.5 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:25.868 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:25.805+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:25.868 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:25.805+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:26.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:25.878+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:26.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:25.868+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:26.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:25.868+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:26.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:25.967+0000 7f961d0db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:26.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:25.967+0000 7f961d0db700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-23T23:48:26.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:25 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:25.967+0000 7f961d0db700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:26.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:26.146+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:26.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:26.146+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:26.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:26.146+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:26.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:26.109+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:26.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:26.109+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:26.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:26 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:26.109+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:26.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:26 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:26.269+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:26.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:26 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:26.248+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:26.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:26 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:26.248+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:27.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:26.838+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:27.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:26.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:27.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:26.890+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:27.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:26.827+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:27.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:26.827+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:27.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:26 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:26.827+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.011850+0000 front 2024-03-23T23:48:03.011824+0000 (oldest deadline 2024-03-23T23:48:25.912029+0000) 2024-03-23T23:48:27.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:27.061+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:27.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:27.061+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:27.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:27.061+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:27.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:27.130+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:27.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:27.130+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:27.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:27 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:27.130+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:27.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:27 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:27.310+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:27.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:27 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:27.310+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:27.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:27 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:27.258+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:27.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:27 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:27.258+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:28.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:27.824+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:28.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:27.824+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:28.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:27.801+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:28.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:27.801+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:28.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:27.801+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.011850+0000 front 2024-03-23T23:48:03.011824+0000 (oldest deadline 2024-03-23T23:48:25.912029+0000) 2024-03-23T23:48:28.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:27.874+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:28.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:27.874+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:28.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:27 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:27.874+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:28.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:28.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:28.665 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:28.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:28.665 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:28.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:28.666 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:28.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:28.666 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:28.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:28.666 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:28 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:28.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:28.667 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:28.261+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:28.667 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:28.261+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:28.667 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:28.283+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:28.667 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:28 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:28.283+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:29.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:28.839+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:29.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:28.839+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:29.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:28.842+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:29.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:28.842+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:29.174 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:28.842+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.011850+0000 front 2024-03-23T23:48:03.011824+0000 (oldest deadline 2024-03-23T23:48:25.912029+0000) 2024-03-23T23:48:29.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:28.879+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:29.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:28.879+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:29.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:28 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:28.879+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:29.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:29.114+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:29.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:29.114+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:29.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:29.114+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:29.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:29.015+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:29.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:29.015+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:29.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:29.015+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:29.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:29.265+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:29.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:29.265+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:29.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:29.258+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:29.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:29.258+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:29.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:29 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:29.258+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:30.074 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:29.850+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:30.074 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:29.850+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:29.803+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:29.803+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:29.803+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.011850+0000 front 2024-03-23T23:48:03.011824+0000 (oldest deadline 2024-03-23T23:48:25.912029+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:29.886+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:29.886+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:30.075 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:29 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:29.886+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:30.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:30.083+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:30.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:30.083+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:30.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:30.083+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:30.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:30.037+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:30.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:30.037+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:30.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:30 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:30.037+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:30.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:30 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:30.221+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:30.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:30 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:30.221+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:30.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:30 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:30.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:30.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:30 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:30.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:30.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:30 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:30.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:31.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:30.810+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:41.809969+0000 front 2024-03-23T23:47:41.809714+0000 (oldest deadline 2024-03-23T23:48:11.810288+0000) 2024-03-23T23:48:31.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:30.810+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.810720+0000 front 2024-03-23T23:47:51.810753+0000 (oldest deadline 2024-03-23T23:48:17.710775+0000) 2024-03-23T23:48:31.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5[48684]: 2024-03-23T23:48:30.810+0000 7f96156c7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.011850+0000 front 2024-03-23T23:48:03.011824+0000 (oldest deadline 2024-03-23T23:48:25.912029+0000) 2024-03-23T23:48:31.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:30.844+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:31.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:30.844+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:31.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:30.908+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:31.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:30.908+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:31.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:30 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:30.908+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:31.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:31.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:31.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:31.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:31.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:31.027+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:31.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:31.096+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:31.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:31.096+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:31.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:31 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:31.096+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:31.585 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:31 smithi174 podman[71628]: 2024-03-23 23:48:31.294631238 +0000 UTC m=+5.525380516 container died 1df85964461040cd927f5eddf69449143f320eee8808e6c470896a11714cc874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:31.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:31 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:31.255+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:31.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:31 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:31.255+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:31.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:31 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:31.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:31.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:31 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:31.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:31.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:31 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:31.240+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:31.901 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:31.828+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:31.902 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:31.828+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:31.902 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:31 smithi174 podman[71628]: 2024-03-23 23:48:31.676143002 +0000 UTC m=+5.906892271 container cleanup 1df85964461040cd927f5eddf69449143f320eee8808e6c470896a11714cc874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-23T23:48:31.902 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:31 smithi174 bash[71628]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5 2024-03-23T23:48:32.173 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:31 smithi174 podman[71657]: 2024-03-23 23:48:31.969482444 +0000 UTC m=+0.673807363 container remove 1df85964461040cd927f5eddf69449143f320eee8808e6c470896a11714cc874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-23T23:48:32.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:31.901+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:32.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:31.901+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:32.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:31 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:31.901+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:32.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:32.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:32.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:32.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:32.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:32.102+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:32.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:32.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:32.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:32.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:32.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:32 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:32.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:32.581 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.183854658 +0000 UTC m=+0.017552443 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:32.581 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.30381939 +0000 UTC m=+0.137517168 container create 007e9ab3c1b266cdae8039b6133b214f6084e8c4e424ec7569385304a7b8a421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:32.581 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.581760045 +0000 UTC m=+0.415457831 container init 007e9ab3c1b266cdae8039b6133b214f6084e8c4e424ec7569385304a7b8a421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-23T23:48:32.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:32 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:32.284+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:32.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:32 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:32.284+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:32.644 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:32 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:32.253+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:32.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:32 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:32.253+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:32.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:32 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:32.253+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:32.845 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.584924389 +0000 UTC m=+0.418622178 container start 007e9ab3c1b266cdae8039b6133b214f6084e8c4e424ec7569385304a7b8a421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:48:32.845 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.673207289 +0000 UTC m=+0.506905091 container attach 007e9ab3c1b266cdae8039b6133b214f6084e8c4e424ec7569385304a7b8a421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-23T23:48:33.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:32.844+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:33.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:32.844+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:33.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:32.936+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:33.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:32.936+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:33.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:32 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:32.936+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:33.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 23 23:48:32 smithi174 podman[71750]: 2024-03-23 23:48:32.872904221 +0000 UTC m=+0.706602022 container died 007e9ab3c1b266cdae8039b6133b214f6084e8c4e424ec7569385304a7b8a421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-23T23:48:33.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:33.065+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:33.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:33.065+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:33.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:33.065+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:33.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:33.036+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:33.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:33.036+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:33.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:33 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:33.036+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:33.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.5.service' 2024-03-23T23:48:33.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:33.492 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-23T23:48:33.492 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-23T23:48:33.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.6 2024-03-23T23:48:33.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:33.272+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:33.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:33.272+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:33.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:33.206+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:33.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:33.206+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:33.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:33 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:33.206+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:33.819 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:33.819+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:33.820 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:33.819+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:33.861 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:33 smithi099 systemd[1]: Stopping Ceph osd.6 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:34.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:33.945+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:34.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:33.945+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:34.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:33 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:33.945+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:34.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:34.031+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:34.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:34.031+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:34.031+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.046+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.046+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.046+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.046+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.066+0000 7fe87bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.066+0000 7fe87bc00700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-23T23:48:34.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:34 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:34.066+0000 7fe87bc00700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:34.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:34.263+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:34.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:34.263+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:34.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:34.193+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:34.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:34.193+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:34.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:34 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:34.193+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:35.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:34.804+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:35.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:34.804+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:35.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:34.918+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:35.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:34.918+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:35.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:34 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:34.918+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:35.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:35.008+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:35.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:35.008+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:35.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:35.008+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:35.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:35.040+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:35.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:35.040+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:35.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:35.040+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:35.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:35.040+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:35.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:35.300+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:35.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:35.300+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:35.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:35.215+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:35.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:35.215+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:35.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:35.215+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:35.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:35 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:35.215+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:36.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:35.794+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:36.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:35.794+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:36.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:35.890+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:36.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:35.890+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:36.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:35 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:35.890+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:36.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.001+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:36.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.001+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:36.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.001+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:36.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.001+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:36.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:36.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:36.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.001+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:36.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:36.291+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:36.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:36.291+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:36.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:36.291+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:36.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:36.165+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:36.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:36.165+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:36.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:36.165+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:36.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:36 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:36.165+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:37.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:36.793+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:37.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:36.793+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:37.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:36.862+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:37.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:36.862+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:37.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:36 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:36.862+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:37.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.962+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:37.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.962+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:37.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.962+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:37.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:36.962+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:37.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.958+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:37.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.958+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:37.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.958+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:37.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:36 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:36.958+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:37.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:37.305+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:37.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:37.305+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:37.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:37.305+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:37.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:37.171+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:37.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:37.171+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:37.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:37.171+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:37.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:37 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:37.171+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:38.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:37.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:38.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:37.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:38.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:37.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:38.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:37.824+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:38.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:37.824+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:38.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:37 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:37.824+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:38.263 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:37.995+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:38.263 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:37.995+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:37.995+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:37.995+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:37.963+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:37.963+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:37.963+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:38.264 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:37 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:37.963+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:38.647 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:38.273+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:38.647 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:38.273+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:38.647 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:38.273+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:38.648 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:38.166+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:38.648 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:38.166+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:38.648 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:38.166+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:38.648 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:38 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:38.166+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:38.778+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:38.778+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:38.778+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:38.820+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:38.820+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:39.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:38 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:38.820+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:39.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:39.000+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.894234+0000 front 2024-03-23T23:47:46.894189+0000 (oldest deadline 2024-03-23T23:48:09.194242+0000) 2024-03-23T23:48:39.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:39.000+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:54.397685+0000 front 2024-03-23T23:47:54.397748+0000 (oldest deadline 2024-03-23T23:48:20.297941+0000) 2024-03-23T23:48:39.361 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:39.000+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:04.202347+0000 front 2024-03-23T23:48:04.202283+0000 (oldest deadline 2024-03-23T23:48:25.902522+0000) 2024-03-23T23:48:39.362 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6[57941]: 2024-03-23T23:48:39.000+0000 7fe877400700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:11.106262+0000 front 2024-03-23T23:48:11.106051+0000 (oldest deadline 2024-03-23T23:48:34.005724+0000) 2024-03-23T23:48:39.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:38 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:38.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:39.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:38 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:38.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:39.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:38 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:38.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:39.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:38 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:38.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:39.546 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:39.313+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:39.313+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:39.313+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:39.147+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:39.147+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:39.147+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:39.547 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:39 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:39.147+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:39.748 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 podman[88992]: 2024-03-23 23:48:39.396859401 +0000 UTC m=+5.480168007 container died 25853cc901c1ad9c662ca5cb1dfde80860b3cd3e24d73f5e97404935468da3c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-23T23:48:39.999 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 podman[88992]: 2024-03-23 23:48:39.960266101 +0000 UTC m=+6.043574736 container cleanup 25853cc901c1ad9c662ca5cb1dfde80860b3cd3e24d73f5e97404935468da3c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:48:39.999 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:39 smithi099 bash[88992]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6 2024-03-23T23:48:40.000 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:39.930+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:40.000 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:39.930+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:40.000 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:39.930+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:40.000 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:39 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:39.930+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:40.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:39.757+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:40.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:39.757+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:40.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:39.757+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:40.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:39.848+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:40.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:39.848+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:40.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:39 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:39.848+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:40.611 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:40 smithi099 podman[89019]: 2024-03-23 23:48:40.245298235 +0000 UTC m=+0.840189022 container remove 25853cc901c1ad9c662ca5cb1dfde80860b3cd3e24d73f5e97404935468da3c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-23T23:48:40.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:40.270+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:40.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:40.270+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:40.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:40.270+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:40.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:40.173+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:40.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:40.173+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:40.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:40.173+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:40.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:40 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:40.173+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:41.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:40.836+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:41.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:40.836+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:41.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:40.836+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:41.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:40.721+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:41.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:40.721+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:41.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:40 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:40.721+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:41.207 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:40 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:40.944+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:41.207 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:40 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:40.944+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:41.207 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:40 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:40.944+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:41.207 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:40 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:40.944+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:41.208 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:40.941149169 +0000 UTC m=+0.033315327 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:41.208 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:41.079629785 +0000 UTC m=+0.171795979 container create c51d0ea1655930282be814b8810d059a63c57c7c265ceee30af24b27ec5d620f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-23T23:48:41.465 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:41.351248391 +0000 UTC m=+0.443414635 container init c51d0ea1655930282be814b8810d059a63c57c7c265ceee30af24b27ec5d620f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5) 2024-03-23T23:48:41.465 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:41.363565528 +0000 UTC m=+0.455731729 container start c51d0ea1655930282be814b8810d059a63c57c7c265ceee30af24b27ec5d620f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T23:48:41.465 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:41.465576814 +0000 UTC m=+0.557743013 container attach c51d0ea1655930282be814b8810d059a63c57c7c265ceee30af24b27ec5d620f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T23:48:41.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:41.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:41.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:41.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:41.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:41.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:41.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:41.144+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:41.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:41.144+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:41.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:41.144+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:41.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:41 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:41.144+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:42.112 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 23 23:48:41 smithi099 podman[89106]: 2024-03-23 23:48:41.791942511 +0000 UTC m=+0.884108699 container died c51d0ea1655930282be814b8810d059a63c57c7c265ceee30af24b27ec5d620f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:42.112 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:41.978+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:42.112 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:41.978+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:42.112 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:41.978+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:42.112 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:41.978+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:42.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:41.860+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:42.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:41.860+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:42.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:41.860+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:42.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:41.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:42.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:41.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:42.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:41 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:41.745+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:42.394 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:42.233+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:42.394 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:42.233+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:42.394 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:42.233+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:42.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:42.110+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:42.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:42.110+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:42.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:42.110+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:42.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:42 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:42.110+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:42.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.6.service' 2024-03-23T23:48:42.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:42.703 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-23T23:48:42.703 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-23T23:48:42.703 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.7 2024-03-23T23:48:43.016 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:42 smithi149 systemd[1]: Stopping Ceph osd.7 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:43.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:42.856+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:43.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:42.856+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:43.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:42.856+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:42.772+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:43.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:42.772+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:43.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:42 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:42.772+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:43.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:42.945+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:43.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:42.945+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:43.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:42.945+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:43.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:42 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:42.945+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:43.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:43.155+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:43.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:43.155+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:43.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:43.155+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:43.155+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.016+0000 7f61080e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.016+0000 7f61080e8700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.016+0000 7f61080e8700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:43.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:43 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:43.226+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:44.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:43.742+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:44.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:43.742+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:44.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:43.742+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:44.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:43.881+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:44.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:43.881+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:44.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:43 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:43.881+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:44.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:43 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:43.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:44.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:43 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:43.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:44.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:43 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:43.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:44.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:43 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:43.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:44.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:44.198+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:44.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:44.198+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:44.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:44.198+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:44.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:44.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:44.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:44.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:44.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:44.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:44.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:44 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:44.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:45.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:44.722+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:45.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:44.722+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:45.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:44.722+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:45.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:44.722+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:45.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:44.912+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:45.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:44.912+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:45.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:44.912+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:45.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:44 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:44.912+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:45.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.011+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:45.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.011+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:45.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.011+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:45.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.011+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:45.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:45.210+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:45.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:45.210+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:45.645 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:45.210+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:45.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:45.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:45.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:45.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:45.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:45.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:45.646 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:45.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:45.646 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:45 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:45.167+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:45.953 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:45.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:45.953 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:45.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:45.954 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:45.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:45.954 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:45.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:46.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.979+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:46.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.979+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:46.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.979+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:46.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:45 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:45.979+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:46.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:45.953+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:46.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:45.953+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:46.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:45.953+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:46.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:45 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:45.953+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:46.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:46.192+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:46.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:46.192+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:46.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:46.192+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:46.644 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:46.192+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.426634+0000 front 2024-03-23T23:48:21.426683+0000 (oldest deadline 2024-03-23T23:48:46.126739+0000) 2024-03-23T23:48:46.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:46.148+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:46.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:46.148+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:46.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:46.148+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:46.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:46.148+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:46.645 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:46 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:46.148+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:46.995 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:46.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:46.995 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:46.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:46.995 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:46.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:46.995 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:46.675+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:47.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:46 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:46.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:47.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:46 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:46.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:47.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:46 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:46.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:47.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:46 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:46.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:47.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:46 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:46.973+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:47.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:47.104+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:47.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:47.104+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:47.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:47.104+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:47.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:47.104+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:47.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:47.104+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:47.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:47.144+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:53.621131+0000 front 2024-03-23T23:47:53.621261+0000 (oldest deadline 2024-03-23T23:48:21.822004+0000) 2024-03-23T23:48:47.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:47.144+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.523084+0000 front 2024-03-23T23:48:03.523069+0000 (oldest deadline 2024-03-23T23:48:27.023073+0000) 2024-03-23T23:48:47.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:47.144+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.725338+0000 front 2024-03-23T23:48:12.725299+0000 (oldest deadline 2024-03-23T23:48:36.225242+0000) 2024-03-23T23:48:47.395 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7[54056]: 2024-03-23T23:48:47.144+0000 7f61006d4700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.426634+0000 front 2024-03-23T23:48:21.426683+0000 (oldest deadline 2024-03-23T23:48:46.126739+0000) 2024-03-23T23:48:47.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:46.995+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:47.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:46.995+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:47.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:46.995+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:47.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:46 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:46.995+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:48.042 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:47 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:47.700+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:48.043 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:47 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:47.700+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:48.043 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:47 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:47.700+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:48.043 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:47 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:47.700+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:48.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:48 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:48.022+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:48.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:48 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:48.022+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:48.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:48 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:48.022+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:48.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:48 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:48.022+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:48.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:48 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:48.022+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:48.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:48 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:48.075+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:48.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:48 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:48.075+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:48.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:48 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:48.075+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:48.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:48 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:48.075+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:48.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:48 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:48.075+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:48.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:48.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:48.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:48.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:48.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:48.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:48.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:48.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:48.759 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:48 smithi149 podman[69989]: 2024-03-23 23:48:48.489487893 +0000 UTC m=+5.589712621 container died e176ed49bad85271a6acce419f592a8f339ebc7946f96cf1b6292d95310b18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-23T23:48:49.030 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:48.740+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:49.030 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:48.740+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:49.030 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:48.740+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:49.030 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:48 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:48.740+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:49.072 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:48 smithi149 podman[69989]: 2024-03-23 23:48:48.859010962 +0000 UTC m=+5.959235687 container cleanup e176ed49bad85271a6acce419f592a8f339ebc7946f96cf1b6292d95310b18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-03-23T23:48:49.072 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:48 smithi149 bash[69989]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7 2024-03-23T23:48:49.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:49 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:49.058+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:49.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:49 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:49.058+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:49.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:49 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:49.058+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:49.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:49 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:49.058+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:49.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:49 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:49.058+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:49.394 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70006]: 2024-03-23 23:48:49.109912516 +0000 UTC m=+0.614248434 container remove e176ed49bad85271a6acce419f592a8f339ebc7946f96cf1b6292d95310b18e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:48:49.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:49.071+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:49.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:49.071+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:49.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:49.071+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:49.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:49.071+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:49.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:49.071+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:49.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:49.030+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:49.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:49.030+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:49.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:49.030+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:49.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:49.030+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:49.771 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.363624717 +0000 UTC m=+0.017826035 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:49.771 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.460994602 +0000 UTC m=+0.115195926 container create d71994722f67ee7ad2553c65965bfe89f56514fbc31dab63bc052d8fd0582ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T23:48:49.771 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.671607009 +0000 UTC m=+0.325808327 container init d71994722f67ee7ad2553c65965bfe89f56514fbc31dab63bc052d8fd0582ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:49.771 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.674615078 +0000 UTC m=+0.328816383 container start d71994722f67ee7ad2553c65965bfe89f56514fbc31dab63bc052d8fd0582ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:49.771 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.771636339 +0000 UTC m=+0.425837681 container attach d71994722f67ee7ad2553c65965bfe89f56514fbc31dab63bc052d8fd0582ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-23T23:48:50.042 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:49.693+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:50.042 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:49.693+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:50.042 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:49.693+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:50.042 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:49 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:49.693+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:50.103 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 23 23:48:49 smithi149 podman[70096]: 2024-03-23 23:48:49.963465707 +0000 UTC m=+0.617667003 container died d71994722f67ee7ad2553c65965bfe89f56514fbc31dab63bc052d8fd0582ecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-23T23:48:50.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:50 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:50.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:50.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:50 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:50.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:50.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:50 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:50.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:50.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:50 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:50.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:50.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:50 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:50.062+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:50.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:50 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:50.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:50.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:50 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:50.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:50.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:50 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:50.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:50.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:50 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:50.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:50.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:50 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:50.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:50.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:50.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:50.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:50.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.042+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:50.599 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.7.service' 2024-03-23T23:48:50.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:50.640 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-23T23:48:50.640 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-23T23:48:50.641 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.8 2024-03-23T23:48:50.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.669+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:50.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.669+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:50.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.669+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:50.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.669+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:50.924 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 systemd[1]: Stopping Ceph osd.8 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:51.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:51 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:51.094+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:51.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:51 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:51.094+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:51.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:51 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:51.094+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:51.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:51 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:51.094+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:51.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:51 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:51.094+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:51.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:51 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:51.094+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:51.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:51 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:51.094+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:51.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:51 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:51.094+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:51.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:51 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:51.094+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:51.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:51 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:51.094+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:51.423 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.945+0000 7f0105f65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:51.423 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.945+0000 7f0105f65700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-23T23:48:51.423 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:50.945+0000 7f0105f65700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:51.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.993+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:51.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.993+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:51.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.993+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:51.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:50 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:50.993+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:51.975 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:51.710+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:51.976 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:51.710+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:51.976 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:51.710+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:51.976 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:51.710+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:52.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:52.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:52.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:52.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:52.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:52.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:52.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:52.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:52.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:52 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:52.103+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:52.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:51.975+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:52.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:51.975+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:52.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:51.975+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:52.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:51.975+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:52.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:51 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:51.975+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:52.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:52 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:52.123+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:52.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:52 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:52.123+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:52.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:52 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:52.123+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:52.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:52 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:52.123+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:52.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:52 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:52.123+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:52.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:52.670+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:52.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:52.670+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:52.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:52.670+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:52.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:52.670+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:53.361 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:53 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:53.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:53.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:53 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:53.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:53.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:53 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:53.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:53.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:53 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:53.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:53.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:53 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:53.089+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:53.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:53.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:53.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:53.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:53.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:53.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:53.069+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:53.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:52.937+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:53.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:52.937+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:53.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:52.937+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:53.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:52.937+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:53.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:52 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:52.937+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:53.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:53.643+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:53.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:53.643+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:53.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:53.643+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:53.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:53.643+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:53.923 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:53.643+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:29.068803+0000 front 2024-03-23T23:48:29.068820+0000 (oldest deadline 2024-03-23T23:48:53.168948+0000) 2024-03-23T23:48:53.924 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:53.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:53.924 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:53.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:53.924 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:53.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:53.924 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:53.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:53.924 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:53 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:53.889+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:54.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:54.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:54.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:54.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:54.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:54.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:54 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:54.109+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:54.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:54 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:54.125+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:54.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:54 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:54.125+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:54.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:54 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:54.125+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:54.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:54 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:54.125+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:54.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:54 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:54.125+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:54.914 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:54.642+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:54.914 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:54.642+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:54.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:54.642+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:54.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:54.642+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:54.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:54.642+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:29.068803+0000 front 2024-03-23T23:48:29.068820+0000 (oldest deadline 2024-03-23T23:48:53.168948+0000) 2024-03-23T23:48:55.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:54.914+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:55.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:54.914+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:55.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:54.914+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:55.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:54.914+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:55.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:54 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:54.914+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:55.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:55.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:55.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:55.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:55.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:55.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:55 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:55.089+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:55.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:55.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:55.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:55.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:55.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:55.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:55 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:55.167+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:48:55.919 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:55.612+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:55.263483+0000 front 2024-03-23T23:47:55.263550+0000 (oldest deadline 2024-03-23T23:48:21.163530+0000) 2024-03-23T23:48:55.919 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:55.612+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:01.664494+0000 front 2024-03-23T23:48:01.664542+0000 (oldest deadline 2024-03-23T23:48:26.964616+0000) 2024-03-23T23:48:55.919 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:55.612+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:12.766100+0000 front 2024-03-23T23:48:12.766479+0000 (oldest deadline 2024-03-23T23:48:36.866022+0000) 2024-03-23T23:48:55.919 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:55.612+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:18.567175+0000 front 2024-03-23T23:48:18.567109+0000 (oldest deadline 2024-03-23T23:48:44.467127+0000) 2024-03-23T23:48:55.919 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8[55732]: 2024-03-23T23:48:55.612+0000 7f0102559700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:29.068803+0000 front 2024-03-23T23:48:29.068820+0000 (oldest deadline 2024-03-23T23:48:53.168948+0000) 2024-03-23T23:48:56.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:55.919+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:56.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:55.919+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:56.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:55.919+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:56.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:55.919+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:56.173 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:55 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:55.919+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:56.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:56.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:56.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:56.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:56.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:56.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:56 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:56.080+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:56.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:56.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:56.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:56.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:56.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:56.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:56 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:56.213+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:48:56.673 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:56 smithi174 podman[71856]: 2024-03-23 23:48:56.423515193 +0000 UTC m=+5.597571448 container died 3afa35e544ce1c599ff889c4df81691b4156374bcccc297d67c5ca24fc95af89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:48:57.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:56 smithi174 podman[71856]: 2024-03-23 23:48:56.730959012 +0000 UTC m=+5.905015258 container cleanup 3afa35e544ce1c599ff889c4df81691b4156374bcccc297d67c5ca24fc95af89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-23T23:48:57.173 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:56 smithi174 bash[71856]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:56 smithi174 podman[71873]: 2024-03-23 23:48:56.957391465 +0000 UTC m=+0.526840985 container remove 3afa35e544ce1c599ff889c4df81691b4156374bcccc297d67c5ca24fc95af89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:56 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:56.943+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:56 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:56.943+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:56 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:56.943+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:56 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:56.943+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:57.174 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:56 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:56.943+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:57.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:57.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:57.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:57.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:57.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:57.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:57.083+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:57.518 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.156972503 +0000 UTC m=+0.017627007 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:48:57.518 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.291580387 +0000 UTC m=+0.152234884 container create 92c5df52800b9b935f9c34b7d8f4c78145ee47f0ac4fca38be02715197c045eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:57.518 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.519032854 +0000 UTC m=+0.379687376 container init 92c5df52800b9b935f9c34b7d8f4c78145ee47f0ac4fca38be02715197c045eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:57.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:57 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:57.257+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:48:57.808 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.522519759 +0000 UTC m=+0.383174256 container start 92c5df52800b9b935f9c34b7d8f4c78145ee47f0ac4fca38be02715197c045eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:48:57.808 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.627395125 +0000 UTC m=+0.488049644 container attach 92c5df52800b9b935f9c34b7d8f4c78145ee47f0ac4fca38be02715197c045eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T23:48:58.105 INFO:journalctl@ceph.osd.8.smithi174.stdout:Mar 23 23:48:57 smithi174 podman[71964]: 2024-03-23 23:48:57.808986872 +0000 UTC m=+0.669641378 container died 92c5df52800b9b935f9c34b7d8f4c78145ee47f0ac4fca38be02715197c045eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-23T23:48:58.105 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:57 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:57.967+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:58.105 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:57 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:57.967+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:58.105 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:57 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:57.967+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:58.106 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:57 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:57.967+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:58.106 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:57 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:57.967+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:48:58.390 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.8.service' 2024-03-23T23:48:58.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:58.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:58.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:58.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:58.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:58.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:58 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:58.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:58.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:48:58.432 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-23T23:48:58.432 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-23T23:48:58.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.9 2024-03-23T23:48:58.518 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:58.518 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:58.519 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:58.519 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:58.519 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:58.519 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:58.240+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:48:58.861 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:58 smithi099 systemd[1]: Stopping Ceph osd.9 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:48:59.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.018+0000 7f3c46c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:48:59.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.018+0000 7f3c46c00700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-23T23:48:59.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.018+0000 7f3c46c00700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:48:59.362 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:48:59.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:48:59.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:48:59.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:48:59.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:48:59.363 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:48:59 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:48:59.227+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:48:59.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:48:59.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:48:59.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:48:59.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:48:59.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:48:59.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:48:59 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:48:59.052+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:48:59.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:58 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:58.957+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:48:59.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:58 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:58.957+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:48:59.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:58 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:58.957+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:48:59.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:58 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:58.957+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:48:59.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:58 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:58.957+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:00.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:00.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:00.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:00.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:00.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:00.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:00 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:00.087+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:00.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:00.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:00.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:00.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:00.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:00.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:48:59 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:48:59.932+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:00.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:49:00.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:49:00.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:49:00.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:49:00.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:49:00.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:00 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:00.209+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:49:01.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:01.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:01.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:01.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:01.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:01.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:01 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:01.044+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:01.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:01.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:01.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:01.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:01.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:01.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:00 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:00.950+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:01.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:49:01.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:49:01.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:49:01.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:49:01.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:49:01.613 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:01 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:01.208+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:49:02.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:02.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:02.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:02.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:02.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:02.396 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:02 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:02.042+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:02.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:02.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:02.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:02.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:02.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:02.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:01.948+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:02.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:49:02.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:49:02.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:49:02.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:49:02.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:49:02.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:02 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:02.235+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:49:03.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:03.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:03.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:03.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:03.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:03.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:03.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:03 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:03.074+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:03.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:02 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:02.961+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.476471+0000 front 2024-03-23T23:47:46.476571+0000 (oldest deadline 2024-03-23T23:48:09.376317+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.379268+0000 front 2024-03-23T23:47:56.379485+0000 (oldest deadline 2024-03-23T23:48:18.678849+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.380772+0000 front 2024-03-23T23:48:03.380890+0000 (oldest deadline 2024-03-23T23:48:25.680972+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:10.283907+0000 front 2024-03-23T23:48:10.283962+0000 (oldest deadline 2024-03-23T23:48:36.183425+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:22.586328+0000 front 2024-03-23T23:48:22.586386+0000 (oldest deadline 2024-03-23T23:48:46.086260+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:27.188588+0000 front 2024-03-23T23:48:27.188144+0000 (oldest deadline 2024-03-23T23:48:54.189199+0000) 2024-03-23T23:49:03.612 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:03 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9[64677]: 2024-03-23T23:49:03.216+0000 7f3c42400700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:36.992212+0000 front 2024-03-23T23:48:36.992312+0000 (oldest deadline 2024-03-23T23:49:02.892294+0000) 2024-03-23T23:49:04.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:04.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:04.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:04.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:04.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:04.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:04.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:04 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:04.067+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:04.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:04.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:04.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:04.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:04.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:04.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:04 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:04.010+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:04.750 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:04 smithi099 podman[89210]: 2024-03-23 23:49:04.433250711 +0000 UTC m=+5.559825207 container died 62e3876d12e5225ea7fdd5c417efd698408d3b27f3fb760e44a185c2e81e55d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-23T23:49:05.111 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:04 smithi099 podman[89210]: 2024-03-23 23:49:04.898774612 +0000 UTC m=+6.025349114 container cleanup 62e3876d12e5225ea7fdd5c417efd698408d3b27f3fb760e44a185c2e81e55d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-23T23:49:05.112 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:04 smithi099 bash[89210]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9 2024-03-23T23:49:05.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:05.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:05.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:05.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:05.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:05.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:05.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:05 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:05.030+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:05.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:05.019+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:05.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:05 smithi099 podman[89228]: 2024-03-23 23:49:05.121606017 +0000 UTC m=+0.675158514 container remove 62e3876d12e5225ea7fdd5c417efd698408d3b27f3fb760e44a185c2e81e55d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True) 2024-03-23T23:49:06.309 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:05 smithi099 podman[89321]: 2024-03-23 23:49:05.888611713 +0000 UTC m=+0.033247562 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:49:06.310 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:06 smithi099 podman[89321]: 2024-03-23 23:49:06.005715821 +0000 UTC m=+0.150351733 container create 696a636355b2c093f34f35cdd79c51e680e1230628967be1568f73dd4e42a594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:49:06.310 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:06 smithi099 podman[89321]: 2024-03-23 23:49:06.309327601 +0000 UTC m=+0.453963493 container init 696a636355b2c093f34f35cdd79c51e680e1230628967be1568f73dd4e42a594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:49:06.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:06.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:06.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:06.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:06.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:06.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:06.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:06 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:06.031+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:06.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:06 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:06.060+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:06.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:06 smithi099 podman[89321]: 2024-03-23 23:49:06.31962993 +0000 UTC m=+0.464265839 container start 696a636355b2c093f34f35cdd79c51e680e1230628967be1568f73dd4e42a594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:49:06.611 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:06 smithi099 podman[89321]: 2024-03-23 23:49:06.408247242 +0000 UTC m=+0.552883107 container attach 696a636355b2c093f34f35cdd79c51e680e1230628967be1568f73dd4e42a594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-23T23:49:07.088 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 23 23:49:06 smithi099 podman[89353]: 2024-03-23 23:49:06.810048629 +0000 UTC m=+0.036321775 container died 696a636355b2c093f34f35cdd79c51e680e1230628967be1568f73dd4e42a594 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-23T23:49:07.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:07.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:07.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:07.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:07.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:07.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:07.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.002+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:07.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:07 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:07.071+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:07.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.9.service' 2024-03-23T23:49:07.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:07.555 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-23T23:49:07.555 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-23T23:49:07.555 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.10 2024-03-23T23:49:07.869 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 systemd[1]: Stopping Ceph osd.10 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:08.144 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.868+0000 7f97b275e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:08.144 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.868+0000 7f97b275e700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-23T23:49:08.144 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:07 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:07.868+0000 7f97b275e700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:08.145 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:08 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:08.013+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:08.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:08.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:08.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:08.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:08.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:08.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:08 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:08.105+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:09.325 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:09.325 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:09.325 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:09.326 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:09.326 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:09.326 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:09.326 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:09 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:09.029+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:09.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:09.099+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:10.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:10.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:10 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:10.026+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:10.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:10 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:10.130+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:11.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:11.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:11 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:11.057+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:11.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:11.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:11.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:11.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:11.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:11.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:11 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:11.082+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:12.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:45.014798+0000 front 2024-03-23T23:47:45.014784+0000 (oldest deadline 2024-03-23T23:48:09.114636+0000) 2024-03-23T23:49:12.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:56.517522+0000 front 2024-03-23T23:47:56.517497+0000 (oldest deadline 2024-03-23T23:48:21.817698+0000) 2024-03-23T23:49:12.394 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:02.919216+0000 front 2024-03-23T23:48:02.918975+0000 (oldest deadline 2024-03-23T23:48:28.818870+0000) 2024-03-23T23:49:12.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-23T23:48:08.819633+0000 front 2024-03-23T23:48:08.819707+0000 (oldest deadline 2024-03-23T23:48:34.719333+0000) 2024-03-23T23:49:12.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:20.520931+0000 front 2024-03-23T23:48:20.520966+0000 (oldest deadline 2024-03-23T23:48:44.621006+0000) 2024-03-23T23:49:12.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:28.121972+0000 front 2024-03-23T23:48:28.121998+0000 (oldest deadline 2024-03-23T23:48:52.221953+0000) 2024-03-23T23:49:12.395 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:12 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10[60661]: 2024-03-23T23:49:12.082+0000 7f97aad4a700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.523325+0000 front 2024-03-23T23:48:37.523291+0000 (oldest deadline 2024-03-23T23:49:02.223032+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:12.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:12 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:12.098+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:13.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:13.072+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:13.501 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:13 smithi149 podman[70206]: 2024-03-23 23:49:13.23146662 +0000 UTC m=+5.483331000 container died f3b2b2d26dd48f773b3e291896cd0117a596c896f132710f53ebaa4576d11b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:49:13.852 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:13 smithi149 podman[70206]: 2024-03-23 23:49:13.58497791 +0000 UTC m=+5.836842280 container cleanup f3b2b2d26dd48f773b3e291896cd0117a596c896f132710f53ebaa4576d11b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-23T23:49:13.852 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:13 smithi149 bash[70206]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10 2024-03-23T23:49:14.144 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:13 smithi149 podman[70236]: 2024-03-23 23:49:13.85264998 +0000 UTC m=+0.618406362 container remove f3b2b2d26dd48f773b3e291896cd0117a596c896f132710f53ebaa4576d11b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:14.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:14 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:14.048+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:14.463 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.104528748 +0000 UTC m=+0.017951303 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:49:14.464 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.203601078 +0000 UTC m=+0.117023623 container create 3e3b256d375c374e0b6e73d392be1aec2031d0ef129c6a9e7c55ad063920542c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-23T23:49:14.464 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.381119691 +0000 UTC m=+0.294542248 container init 3e3b256d375c374e0b6e73d392be1aec2031d0ef129c6a9e7c55ad063920542c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:49:14.464 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.384139361 +0000 UTC m=+0.297561908 container start 3e3b256d375c374e0b6e73d392be1aec2031d0ef129c6a9e7c55ad063920542c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:49:14.464 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.464302528 +0000 UTC m=+0.377725068 container attach 3e3b256d375c374e0b6e73d392be1aec2031d0ef129c6a9e7c55ad063920542c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:49:14.894 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 23 23:49:14 smithi149 podman[70327]: 2024-03-23 23:49:14.679313637 +0000 UTC m=+0.592736184 container died 3e3b256d375c374e0b6e73d392be1aec2031d0ef129c6a9e7c55ad063920542c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True) 2024-03-23T23:49:15.301 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.10.service' 2024-03-23T23:49:15.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:15.343 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-23T23:49:15.343 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-23T23:49:15.344 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.11 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:15.373 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.095+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:15.666 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 systemd[1]: Stopping Ceph osd.11 for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:15.923 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.665+0000 7fcc06172700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:15.923 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.665+0000 7fcc06172700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-23T23:49:15.923 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:15 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:15.665+0000 7fcc06172700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T23:49:16.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:16.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:16.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:16.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:16.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:16.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:16.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:16 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:16.128+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:17.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:17 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:17.158+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:18.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:18 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:18.157+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:19.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:19.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:19.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:19.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:19.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:19.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:19.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:19.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:19 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:19.115+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-03-23T23:48:52.749557+0000 front 2024-03-23T23:48:52.749575+0000 (oldest deadline 2024-03-23T23:49:18.649620+0000) 2024-03-23T23:49:20.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-23T23:47:46.640537+0000 front 2024-03-23T23:47:46.640603+0000 (oldest deadline 2024-03-23T23:48:11.940479+0000) 2024-03-23T23:49:20.423 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-23T23:47:51.941076+0000 front 2024-03-23T23:47:51.941066+0000 (oldest deadline 2024-03-23T23:48:23.141750+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-03-23T23:48:03.142273+0000 front 2024-03-23T23:48:03.142306+0000 (oldest deadline 2024-03-23T23:48:27.842399+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-23T23:48:21.845069+0000 front 2024-03-23T23:48:21.845144+0000 (oldest deadline 2024-03-23T23:48:44.145089+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-03-23T23:48:30.045888+0000 front 2024-03-23T23:48:30.045964+0000 (oldest deadline 2024-03-23T23:48:51.746027+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-23T23:48:37.047488+0000 front 2024-03-23T23:48:37.047351+0000 (oldest deadline 2024-03-23T23:48:59.847439+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-23T23:48:46.849270+0000 front 2024-03-23T23:48:46.849206+0000 (oldest deadline 2024-03-23T23:49:12.749174+0000) 2024-03-23T23:49:20.424 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:20 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11[62346]: 2024-03-23T23:49:20.116+0000 7fcbfe75e700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-03-23T23:48:52.749557+0000 front 2024-03-23T23:48:52.749575+0000 (oldest deadline 2024-03-23T23:49:18.649620+0000) 2024-03-23T23:49:21.382 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:21 smithi174 podman[72080]: 2024-03-23 23:49:21.111953292 +0000 UTC m=+5.576785745 container died fd34b8cb2c6582901bfd446f76fab117a43877b5f51c99d17b51b91b8f9572a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-23T23:49:21.634 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:21 smithi174 podman[72080]: 2024-03-23 23:49:21.449465636 +0000 UTC m=+5.914298097 container cleanup fd34b8cb2c6582901bfd446f76fab117a43877b5f51c99d17b51b91b8f9572a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-23T23:49:21.634 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:21 smithi174 bash[72080]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11 2024-03-23T23:49:21.923 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:21 smithi174 podman[72098]: 2024-03-23 23:49:21.634281275 +0000 UTC m=+0.522497855 container remove fd34b8cb2c6582901bfd446f76fab117a43877b5f51c99d17b51b91b8f9572a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:49:22.311 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:21.95057575 +0000 UTC m=+0.017354069 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-23T23:49:22.312 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:22.076733715 +0000 UTC m=+0.143512016 container create 194b297223dd97e1c8eb0fe22dcfc56364431837ad0831dca1696281c8078975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True) 2024-03-23T23:49:22.312 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:22.312502002 +0000 UTC m=+0.379280310 container init 194b297223dd97e1c8eb0fe22dcfc56364431837ad0831dca1696281c8078975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-23T23:49:22.603 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:22.315593127 +0000 UTC m=+0.382371428 container start 194b297223dd97e1c8eb0fe22dcfc56364431837ad0831dca1696281c8078975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T23:49:22.603 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:22.404110482 +0000 UTC m=+0.470888798 container attach 194b297223dd97e1c8eb0fe22dcfc56364431837ad0831dca1696281c8078975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:49:22.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Mar 23 23:49:22 smithi174 podman[72190]: 2024-03-23 23:49:22.603936833 +0000 UTC m=+0.670715155 container died 194b297223dd97e1c8eb0fe22dcfc56364431837ad0831dca1696281c8078975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T23:49:23.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@osd.11.service' 2024-03-23T23:49:23.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:23.273 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-23T23:49:23.273 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-23T23:49:23.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.a 2024-03-23T23:49:23.611 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:23 smithi099 systemd[1]: Stopping Ceph mds.a for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:24.111 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: 2024-03-23T23:49:23.836+0000 7f97e3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:24.112 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:23 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a[71183]: 2024-03-23T23:49:23.836+0000 7f97e3200700 -1 mds.a *** got signal Terminated *** 2024-03-23T23:49:28.111 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:27 smithi099 podman[89437]: 2024-03-23 23:49:27.857649047 +0000 UTC m=+4.151908633 container died beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:49:28.611 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:28 smithi099 podman[89437]: 2024-03-23 23:49:28.294446121 +0000 UTC m=+4.588705711 container cleanup beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-23T23:49:28.611 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:28 smithi099 bash[89437]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a 2024-03-23T23:49:29.111 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 23 23:49:28 smithi099 podman[89451]: 2024-03-23 23:49:28.612228492 +0000 UTC m=+0.740692948 container remove beb5a84fadda42fdd69e6e8c3fe6c3450aa677d8cfa139c6594d5feb2a99c823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:49:29.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.a.service' 2024-03-23T23:49:29.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:29.361 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-23T23:49:29.361 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-23T23:49:29.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.d 2024-03-23T23:49:29.861 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:29 smithi099 systemd[1]: Stopping Ceph mds.d for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:30.361 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d[70193]: 2024-03-23T23:49:29.979+0000 7fcd23800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:30.361 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:29 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d[70193]: 2024-03-23T23:49:29.979+0000 7fcd23800700 -1 mds.d *** got signal Terminated *** 2024-03-23T23:49:34.178 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:33 smithi099 podman[89556]: 2024-03-23 23:49:33.880915766 +0000 UTC m=+4.054138403 container died 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T23:49:34.549 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:34 smithi099 podman[89556]: 2024-03-23 23:49:34.348236962 +0000 UTC m=+4.521459605 container cleanup 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-23T23:49:34.550 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:34 smithi099 bash[89556]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d 2024-03-23T23:49:34.861 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 23 23:49:34 smithi099 podman[89568]: 2024-03-23 23:49:34.549299118 +0000 UTC m=+0.656333215 container remove 972cf0b8722e48a1cb5406baabb8bd4b1359267fd3abe8e74a6c23332278102b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-23T23:49:35.064 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.d.service' 2024-03-23T23:49:35.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:35.180 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-23T23:49:35.180 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-23T23:49:35.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.g 2024-03-23T23:49:35.611 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:35 smithi099 systemd[1]: Stopping Ceph mds.g for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:36.111 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: 2024-03-23T23:49:35.798+0000 7f381d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:36.111 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:35 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g[70838]: 2024-03-23T23:49:35.798+0000 7f381d000700 -1 mds.g *** got signal Terminated *** 2024-03-23T23:49:40.111 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:39 smithi099 podman[89676]: 2024-03-23 23:49:39.810884505 +0000 UTC m=+4.146309530 container died 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-23T23:49:40.486 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:40 smithi099 podman[89676]: 2024-03-23 23:49:40.259606476 +0000 UTC m=+4.595031472 container cleanup 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-23T23:49:40.486 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:40 smithi099 bash[89676]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g 2024-03-23T23:49:40.861 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 23 23:49:40 smithi099 podman[89689]: 2024-03-23 23:49:40.485448452 +0000 UTC m=+0.661110945 container remove 6964d9a17ad37d7efe903d665ae6960310bde58ece5591a332f06cb0877bf9c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS) 2024-03-23T23:49:41.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.g.service' 2024-03-23T23:49:41.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:41.110 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-23T23:49:41.110 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-23T23:49:41.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.j 2024-03-23T23:49:41.361 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:41 smithi099 systemd[1]: Stopping Ceph mds.j for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:42.114 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j[69850]: 2024-03-23T23:49:41.668+0000 7fb849000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:42.115 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:41 smithi099 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j[69850]: 2024-03-23T23:49:41.668+0000 7fb849000700 -1 mds.j *** got signal Terminated *** 2024-03-23T23:49:46.000 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:45 smithi099 podman[89794]: 2024-03-23 23:49:45.686663766 +0000 UTC m=+4.123611512 container died 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:49:46.362 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:46 smithi099 podman[89794]: 2024-03-23 23:49:46.112809142 +0000 UTC m=+4.549756893 container cleanup 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T23:49:46.362 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:46 smithi099 bash[89794]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j 2024-03-23T23:49:46.818 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 23 23:49:46 smithi099 podman[89820]: 2024-03-23 23:49:46.386836885 +0000 UTC m=+0.684497406 container remove 6365bce062e6b1920f30a5af3d570fdcc4b5c2f7d63372da6337580f799239ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-23T23:49:46.853 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.j.service' 2024-03-23T23:49:46.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:46.974 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-23T23:49:46.974 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-23T23:49:46.974 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.b 2024-03-23T23:49:47.267 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:47 smithi149 systemd[1]: Stopping Ceph mds.b for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:47.534 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:49:47.266+0000 7f0e6a100700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:47.534 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:47 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b[66185]: 2024-03-23T23:49:47.266+0000 7f0e6a100700 -1 mds.b *** got signal Terminated *** 2024-03-23T23:49:48.895 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:48 smithi149 podman[70434]: 2024-03-23 23:49:48.67969158 +0000 UTC m=+1.507854529 container died 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-23T23:49:49.483 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:49 smithi149 podman[70434]: 2024-03-23 23:49:49.258199236 +0000 UTC m=+2.086362178 container cleanup 41a8fa1b155420845066632f3e9405463c594894b1cc399dcb16405386bf1837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-23T23:49:49.483 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 23 23:49:49 smithi149 bash[70434]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-b 2024-03-23T23:49:49.627 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.b.service' 2024-03-23T23:49:49.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:49.666 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-23T23:49:49.667 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-23T23:49:49.667 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.e 2024-03-23T23:49:50.144 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:49 smithi149 systemd[1]: Stopping Ceph mds.e for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:50.144 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: 2024-03-23T23:49:49.986+0000 7fbf7aa38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:50.144 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:49 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e[65569]: 2024-03-23T23:49:49.986+0000 7fbf7aa38700 -1 mds.e *** got signal Terminated *** 2024-03-23T23:49:52.994 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:52 smithi149 podman[70567]: 2024-03-23 23:49:52.67163595 +0000 UTC m=+2.787354261 container died ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-23T23:49:53.303 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:53 smithi149 podman[70567]: 2024-03-23 23:49:53.078081176 +0000 UTC m=+3.193799496 container cleanup ae5b11d1a3ad4a6a5bc2e54becc3c07c1b7191d69455a9ba749ee22d7353f75e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-23T23:49:53.303 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 23 23:49:53 smithi149 bash[70567]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-e 2024-03-23T23:49:53.440 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.e.service' 2024-03-23T23:49:53.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:53.476 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-23T23:49:53.476 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-23T23:49:53.476 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.h 2024-03-23T23:49:53.894 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:53 smithi149 systemd[1]: Stopping Ceph mds.h for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:53.894 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: 2024-03-23T23:49:53.798+0000 7f2aa1427700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:53.894 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:53 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h[65879]: 2024-03-23T23:49:53.798+0000 7f2aa1427700 -1 mds.h *** got signal Terminated *** 2024-03-23T23:49:57.023 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:56 smithi149 podman[70689]: 2024-03-23 23:49:56.688473078 +0000 UTC m=+2.997621395 container died d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-23T23:49:57.307 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 podman[70689]: 2024-03-23 23:49:57.131555039 +0000 UTC m=+3.440703355 container cleanup d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-23T23:49:57.307 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 bash[70689]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h 2024-03-23T23:49:57.549 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.h.service' 2024-03-23T23:49:57.572 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 podman[70702]: 2024-03-23 23:49:57.307662505 +0000 UTC m=+0.613348978 container remove d6fb423a1508e5263577af83224ca7dab00d4b6699d89a92bfb233d19696cd7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-23T23:49:57.572 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.h.service: Deactivated successfully. 2024-03-23T23:49:57.572 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 systemd[1]: Stopped Ceph mds.h for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:49:57.573 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 23 23:49:57 smithi149 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.h.service: Consumed 2.459s CPU time. 2024-03-23T23:49:57.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:49:57.584 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-23T23:49:57.584 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-23T23:49:57.584 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.k 2024-03-23T23:49:57.894 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:49:57 smithi149 systemd[1]: Stopping Ceph mds.k for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:49:58.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:49:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: 2024-03-23T23:49:57.909+0000 7fa7147a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:49:58.394 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:49:57 smithi149 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k[65251]: 2024-03-23T23:49:57.909+0000 7fa7147a4700 -1 mds.k *** got signal Terminated *** 2024-03-23T23:50:01.001 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:00 smithi149 podman[70813]: 2024-03-23 23:50:00.68606324 +0000 UTC m=+2.883428434 container died e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T23:50:01.293 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 podman[70813]: 2024-03-23 23:50:01.101412931 +0000 UTC m=+3.298778116 container cleanup e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-23T23:50:01.293 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 bash[70813]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k 2024-03-23T23:50:01.532 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.k.service' 2024-03-23T23:50:01.556 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 podman[70828]: 2024-03-23 23:50:01.294079186 +0000 UTC m=+0.604409577 container remove e4f25c19cb5ee741d9f57c88f23ad276c1aa5261fb87c611a26e4f4b2cb71fe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-k, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T23:50:01.556 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.k.service: Deactivated successfully. 2024-03-23T23:50:01.556 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 systemd[1]: Stopped Ceph mds.k for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:50:01.556 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 23 23:50:01 smithi149 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.k.service: Consumed 1.300s CPU time. 2024-03-23T23:50:01.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:50:01.567 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-23T23:50:01.568 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-23T23:50:01.568 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.c 2024-03-23T23:50:01.923 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:01 smithi174 systemd[1]: Stopping Ceph mds.c for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:50:02.185 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: 2024-03-23T23:50:01.930+0000 7fd305c0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:50:02.185 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:01 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c[67810]: 2024-03-23T23:50:01.930+0000 7fd305c0e700 -1 mds.c *** got signal Terminated *** 2024-03-23T23:50:04.997 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:04 smithi174 podman[72298]: 2024-03-23 23:50:04.674163776 +0000 UTC m=+2.917464613 container died 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:50:05.281 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 podman[72298]: 2024-03-23 23:50:05.089264904 +0000 UTC m=+3.332565738 container cleanup 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-23T23:50:05.281 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 bash[72298]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c 2024-03-23T23:50:05.507 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.c.service' 2024-03-23T23:50:05.531 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 podman[72324]: 2024-03-23 23:50:05.28211146 +0000 UTC m=+0.607631910 container remove 71029bed2fe85fe76d7b3e3fd334a92b010db6d5fe3bf50b17706d3737f73fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:50:05.531 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.c.service: Deactivated successfully. 2024-03-23T23:50:05.531 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 systemd[1]: Stopped Ceph mds.c for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:50:05.531 INFO:journalctl@ceph.mds.c.smithi174.stdout:Mar 23 23:50:05 smithi174 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.c.service: Consumed 1min 57.351s CPU time. 2024-03-23T23:50:05.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:50:05.546 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-23T23:50:05.546 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-23T23:50:05.546 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.f 2024-03-23T23:50:05.871 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:05 smithi174 systemd[1]: Stopping Ceph mds.f for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:50:06.173 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: 2024-03-23T23:50:05.871+0000 7f1d7d6bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:50:06.173 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:05 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f[67506]: 2024-03-23T23:50:05.871+0000 7f1d7d6bc700 -1 mds.f *** got signal Terminated *** 2024-03-23T23:50:08.951 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:08 smithi174 podman[72430]: 2024-03-23 23:50:08.679760172 +0000 UTC m=+2.918816240 container died 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:50:09.276 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:09 smithi174 podman[72430]: 2024-03-23 23:50:09.034198687 +0000 UTC m=+3.273254744 container cleanup 6c97a7bd053e7e2d06bd74bbd0776fa9da1cf8930e2926f5c41d5ae605485911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T23:50:09.276 INFO:journalctl@ceph.mds.f.smithi174.stdout:Mar 23 23:50:09 smithi174 bash[72430]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-f 2024-03-23T23:50:09.396 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.f.service' 2024-03-23T23:50:09.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:50:09.433 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-23T23:50:09.433 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-23T23:50:09.433 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.i 2024-03-23T23:50:09.673 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:09 smithi174 systemd[1]: Stopping Ceph mds.i for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:50:10.173 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: 2024-03-23T23:50:09.762+0000 7f7f52336700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:50:10.173 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:09 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i[68122]: 2024-03-23T23:50:09.762+0000 7f7f52336700 -1 mds.i *** got signal Terminated *** 2024-03-23T23:50:12.923 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:12 smithi174 podman[72551]: 2024-03-23 23:50:12.680246713 +0000 UTC m=+3.024628490 container died 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-23T23:50:13.191 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:13 smithi174 podman[72551]: 2024-03-23 23:50:13.012579683 +0000 UTC m=+3.356961451 container cleanup 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-23T23:50:13.191 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:13 smithi174 bash[72551]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i 2024-03-23T23:50:13.456 INFO:journalctl@ceph.mds.i.smithi174.stdout:Mar 23 23:50:13 smithi174 podman[72564]: 2024-03-23 23:50:13.238820114 +0000 UTC m=+0.554807728 container remove 1a08a22abf3e012bec5a7479cef9389a9ee7470aa9ad40d96ba57cd62ebe9839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T23:50:13.470 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.i.service' 2024-03-23T23:50:13.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:50:13.505 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-23T23:50:13.505 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-23T23:50:13.505 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.l 2024-03-23T23:50:13.799 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:13 smithi174 systemd[1]: Stopping Ceph mds.l for 5fd03d00-e96d-11ee-95cd-87774f69a715... 2024-03-23T23:50:14.173 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: 2024-03-23T23:50:13.799+0000 7f8835974700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T23:50:14.173 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:13 smithi174 ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l[67191]: 2024-03-23T23:50:13.799+0000 7f8835974700 -1 mds.l *** got signal Terminated *** 2024-03-23T23:50:16.923 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:16 smithi174 podman[72672]: 2024-03-23 23:50:16.671533498 +0000 UTC m=+2.953430593 container died b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T23:50:17.199 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 podman[72672]: 2024-03-23 23:50:17.015622411 +0000 UTC m=+3.297519514 container cleanup b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-23T23:50:17.199 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 bash[72672]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l 2024-03-23T23:50:17.435 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.l.service' 2024-03-23T23:50:17.458 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 podman[72686]: 2024-03-23 23:50:17.199861411 +0000 UTC m=+0.522748926 container remove b85a40263aea2b08e1cd5d3a75e51e389edfa6094921144727cc6d8126716cdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-5fd03d00-e96d-11ee-95cd-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T23:50:17.458 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-23T23:50:17.458 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 systemd[1]: Stopped Ceph mds.l for 5fd03d00-e96d-11ee-95cd-87774f69a715. 2024-03-23T23:50:17.458 INFO:journalctl@ceph.mds.l.smithi174.stdout:Mar 23 23:50:17 smithi174 systemd[1]: ceph-5fd03d00-e96d-11ee-95cd-87774f69a715@mds.l.service: Consumed 1.258s CPU time. 2024-03-23T23:50:17.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T23:50:17.469 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-23T23:50:17.469 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T23:50:17.794 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:22.594 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T23:50:22.785 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:25.222 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T23:50:25.415 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:27.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:50:27.935 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:50:27.969 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T23:50:27.998 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-23T23:50:28.000 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi099/crash 2024-03-23T23:50:28.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash -- . 2024-03-23T23:50:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T23:50:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-03-23T23:50:28.127 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi149/crash 2024-03-23T23:50:28.128 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash -- . 2024-03-23T23:50:28.178 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T23:50:28.178 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-03-23T23:50:28.180 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi174/crash 2024-03-23T23:50:28.181 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash -- . 2024-03-23T23:50:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T23:50:28.216 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-03-23T23:50:28.217 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-23T23:50:28.217 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T23:50:28.320 INFO:teuthology.orchestra.run.smithi099.stdout:2024-03-23T23:34:18.320534+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-23T23:50:28.322 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-23T23:50:28.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[SEC\]' /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T23:50:28.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]' /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T23:50:28.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[WRN\]' /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T23:50:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:2024-03-23T23:34:18.320534+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-23T23:50:28.584 INFO:tasks.cephadm:Compressing logs... 2024-03-23T23:50:28.585 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T23:50:28.589 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T23:50:28.592 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T23:50:28.619 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T23:50:28.619 INFO:teuthology.orchestra.run.smithi174.stderr:: No such file or directory 2024-03-23T23:50:28.619 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-23T23:50:28.619 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T23:50:28.620 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-23T23:50:28.620 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-23T23:50:28.620 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-23T23:50:28.620 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.i.log: /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-23T23:50:28.626 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-23T23:50:28.627 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-23T23:50:28.628 INFO:teuthology.orchestra.run.smithi149.stderr: 90.3%/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.7.log: -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T23:50:28.628 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log 2024-03-23T23:50:28.630 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T23:50:28.631 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-23T23:50:28.631 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log: 86.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T23:50:28.631 INFO:teuthology.orchestra.run.smithi174.stderr: 92.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-23T23:50:28.631 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-23T23:50:28.631 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T23:50:28.637 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T23:50:28.638 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-23T23:50:28.644 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.y.log: 92.3% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-23T23:50:28.649 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T23:50:28.650 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-03-23T23:50:28.650 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-23T23:50:28.651 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-23T23:50:28.652 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T23:50:28.653 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-23T23:50:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-23T23:50:28.656 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-23T23:50:28.659 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.c.log: 94.1% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-23T23:50:28.659 INFO:teuthology.orchestra.run.smithi174.stderr: 90.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T23:50:28.660 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log 2024-03-23T23:50:28.660 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T23:50:28.662 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-23T23:50:28.662 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T23:50:28.662 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.x.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T23:50:28.662 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-23T23:50:28.662 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T23:50:28.663 INFO:teuthology.orchestra.run.smithi099.stderr: 94.6% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-23T23:50:28.663 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log 2024-03-23T23:50:28.665 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T23:50:28.667 INFO:teuthology.orchestra.run.smithi149.stderr: 94.1% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-23T23:50:28.667 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T23:50:28.668 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T23:50:28.668 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-23T23:50:28.668 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log: 86.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T23:50:28.669 INFO:teuthology.orchestra.run.smithi149.stderr: 94.1%gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-23T23:50:28.669 INFO:teuthology.orchestra.run.smithi149.stderr: -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-23T23:50:28.670 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-23T23:50:28.670 INFO:teuthology.orchestra.run.smithi099.stderr: 86.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T23:50:28.671 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-23T23:50:28.676 INFO:teuthology.orchestra.run.smithi174.stderr: 93.2% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-23T23:50:28.676 INFO:teuthology.orchestra.run.smithi174.stderr: 94.2% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-23T23:50:28.679 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.d.log: /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.a.log: /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-23T23:50:28.683 INFO:teuthology.orchestra.run.smithi174.stderr: 93.1% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-23T23:50:28.684 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T23:50:28.686 INFO:teuthology.orchestra.run.smithi099.stderr: 90.8% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T23:50:28.726 INFO:teuthology.orchestra.run.smithi099.stderr: 92.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-23T23:50:28.758 INFO:teuthology.orchestra.run.smithi099.stderr: 92.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-23T23:50:28.890 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-23T23:50:28.899 INFO:teuthology.orchestra.run.smithi174.stderr: 91.8% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-23T23:50:29.020 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-23T23:50:29.081 INFO:teuthology.orchestra.run.smithi149.stderr: 95.7% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-23T23:50:29.342 INFO:teuthology.orchestra.run.smithi099.stderr: 90.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-23T23:50:29.485 INFO:teuthology.orchestra.run.smithi099.stderr: 91.5% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-23T23:50:29.579 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-23T23:50:29.804 INFO:teuthology.orchestra.run.smithi174.stderr: 92.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-23T23:50:29.990 INFO:teuthology.orchestra.run.smithi174.stderr: 90.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-23T23:50:29.991 INFO:teuthology.orchestra.run.smithi149.stderr: 92.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-23T23:50:30.102 INFO:teuthology.orchestra.run.smithi099.stderr: 92.7% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-23T23:50:30.152 INFO:teuthology.orchestra.run.smithi174.stderr: 92.7% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-23T23:50:30.171 INFO:teuthology.orchestra.run.smithi099.stderr: 92.4% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-23T23:50:30.176 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-03-23T23:50:30.176 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m1.557s 2024-03-23T23:50:30.176 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m5.370s 2024-03-23T23:50:30.176 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.495s 2024-03-23T23:50:30.549 INFO:teuthology.orchestra.run.smithi149.stderr: 92.6% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-23T23:50:30.587 INFO:teuthology.orchestra.run.smithi149.stderr: 92.6% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-23T23:50:30.954 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-23T23:50:31.710 INFO:teuthology.orchestra.run.smithi174.stderr: 94.3% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-23T23:50:31.711 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-03-23T23:50:31.711 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m3.106s 2024-03-23T23:50:31.711 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m7.390s 2024-03-23T23:50:31.711 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.260s 2024-03-23T23:50:33.188 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/5fd03d00-e96d-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-23T23:50:33.189 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-23T23:50:33.189 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m4.583s 2024-03-23T23:50:33.189 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m12.634s 2024-03-23T23:50:33.189 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.447s 2024-03-23T23:50:33.195 INFO:tasks.cephadm:Archiving logs... 2024-03-23T23:50:33.196 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi099/log 2024-03-23T23:50:33.200 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T23:50:34.219 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi149/log 2024-03-23T23:50:34.220 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T23:50:36.155 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi174/log 2024-03-23T23:50:36.156 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T23:50:37.338 INFO:tasks.cephadm:Removing cluster... 2024-03-23T23:50:37.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force 2024-03-23T23:50:37.666 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:38.254 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force 2024-03-23T23:50:38.444 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:38.714 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5fd03d00-e96d-11ee-95cd-87774f69a715 --force 2024-03-23T23:50:38.904 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 5fd03d00-e96d-11ee-95cd-87774f69a715 2024-03-23T23:50:39.168 INFO:tasks.cephadm:Removing cephadm ... 2024-03-23T23:50:39.168 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T23:50:39.225 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T23:50:39.240 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T23:50:39.255 INFO:tasks.cephadm:Teardown complete 2024-03-23T23:50:39.256 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T23:50:39.267 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T23:50:39.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T23:50:39.275 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T23:50:39.283 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T23:50:39.444 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T23:50:39.455 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T23:50:39.455 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:50:39.464 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:50:39.466 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T23:50:39.477 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:50:39.479 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:50:39.482 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:50:39.483 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-23T23:50:39.483 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 -263us[ -339us] +/- 103ms 2024-03-23T23:50:39.483 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -836us[ -836us] +/- 59ms 2024-03-23T23:50:39.483 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +1327us[+1251us] +/- 77ms 2024-03-23T23:50:39.483 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 -260us[ -260us] +/- 103ms 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -825us[ -826us] +/- 59ms 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1326us[+1323us] +/- 77ms 2024-03-23T23:50:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:50:39.507 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 88 -257us[ -298us] +/- 102ms 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -842us[ -855us] +/- 59ms 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +1306us[+1291us] +/- 77ms 2024-03-23T23:50:39.517 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T23:50:39.519 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T23:50:39.532 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T23:50:39.532 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T23:50:39.543 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T23:50:39.768 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T23:50:39.839 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T23:50:39.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:50:40.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:40.014 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:50:40.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:40.045 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T23:50:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:40.076 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T23:50:40.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T23:50:40.121 INFO:teuthology.task.internal:Duration was 1675.370806 seconds 2024-03-23T23:50:40.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T23:50:40.131 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T23:50:40.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T23:50:40.139 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T23:50:40.144 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T23:50:40.185 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:50:40.187 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:50:40.296 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T23:50:40.689 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T23:50:40.689 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-03-23T23:50:40.690 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T23:50:40.748 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-03-23T23:50:40.749 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T23:50:40.776 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-03-23T23:50:40.776 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T23:50:40.811 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T23:50:40.811 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T23:50:40.820 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T23:50:40.822 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T23:50:40.907 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T23:50:40.908 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T23:50:40.914 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T23:50:40.917 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T23:50:41.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T23:50:41.424 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T23:50:41.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T23:50:41.521 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T23:50:41.549 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T23:50:41.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T23:50:41.590 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.598 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.601 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.624 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-03-23T23:50:41.645 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-03-23T23:50:41.701 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-03-23T23:50:41.753 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:41.799 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:41.814 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T23:50:41.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T23:50:41.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T23:50:41.841 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T23:50:41.842 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi099 2024-03-23T23:50:41.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T23:50:41.960 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi149 2024-03-23T23:50:41.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T23:50:41.999 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/remote/smithi174 2024-03-23T23:50:41.999 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T23:50:42.040 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T23:50:42.040 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T23:50:42.048 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T23:50:42.050 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T23:50:42.098 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T23:50:42.110 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T23:50:42.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T23:50:42.122 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T23:50:42.122 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T23:50:42.130 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T23:50:42.132 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T23:50:42.144 INFO:teuthology.orchestra.run.smithi149.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 23:50 /home/ubuntu/cephtest 2024-03-23T23:50:42.147 INFO:teuthology.orchestra.run.smithi174.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 23:50 /home/ubuntu/cephtest 2024-03-23T23:50:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 23:50 /home/ubuntu/cephtest 2024-03-23T23:50:42.181 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T23:50:42.194 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T23:50:42.218 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T23:50:42.247 DEBUG:teuthology.nuke:shortname: smithi099 2024-03-23T23:50:42.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T23:50:42.272 DEBUG:teuthology.nuke:shortname: smithi149 2024-03-23T23:50:42.273 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T23:50:42.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.272089', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB4UZn3fzouSCKfvtBoeO0x+s9TZX/BRQVoHoDAsAWjVY37BVvbBlS2hn2A0XKBXK09iBkGlHyCD8k0Q0WOGyQc='} 2024-03-23T23:50:42.296 DEBUG:teuthology.nuke:shortname: smithi174 2024-03-23T23:50:42.296 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T23:50:42.300 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.269533', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2OXbuRcMHP6jEp4VCLlldmKzvVZHylexlT3WdnUEYRmPOnDPk7g3E2ykjebN6SHp8IKLkM9wMhWfbUfmhLemA='} 2024-03-23T23:50:42.327 INFO:teuthology.orchestra.console.smithi099:Power off 2024-03-23T23:50:42.328 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T23:50:42.341 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 23:12:54.270962', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUAe1z2NFyLdfNY3eaJkHruYWyySMxXI+yKD+SaVX2VXOa/5AMqzJVUtF3d2ZWsLcQZDqZgQfb2IqL7gkNTY4Y='} 2024-03-23T23:50:42.351 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2024-03-23T23:50:42.351 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:42.374 INFO:teuthology.orchestra.console.smithi149:Power off 2024-03-23T23:50:42.374 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T23:50:42.383 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-23T23:50:42.395 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-03-23T23:50:42.395 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:42.410 INFO:teuthology.orchestra.console.smithi174:Power off 2024-03-23T23:50:42.411 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T23:50:42.421 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-23T23:50:42.434 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-03-23T23:50:42.434 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:42.461 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-23T23:50:46.384 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:46.422 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:46.463 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:46.495 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-23T23:50:46.532 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-23T23:50:46.576 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-23T23:50:50.499 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:50.533 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:50.577 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:50.610 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2024-03-23T23:50:50.643 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-03-23T23:50:50.690 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-23T23:50:50.712 INFO:teuthology.orchestra.console.smithi099:Power off completed 2024-03-23T23:50:50.744 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-03-23T23:50:54.694 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T23:50:54.804 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-03-23T23:50:54.905 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-03-23T23:50:55.010 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} duration: 1675.3708057403564 failure_reason: '"2024-03-23T23:34:18.320534+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: mchangir success: false 2024-03-23T23:50:55.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T23:50:55.098 INFO:teuthology.run:FAIL